Historical errata for The Definitive ANTLR Reference

The generated lexer grammar is not Expr.g but Expr.gl (by the way I’m using ANTLR 3.0b6 which is currently the last released version so it may be different in the previous or next version)

‘inddicates’ instead of ‘indicates’

It seems the actions are referred to by the fact that it’s the nth ACTION in the whole file. We can guess by the context that action 4 is the action after ID and then what are action 5 and 6, but even with the whole file it’s not so easy to see why you use these numbers because we have tendency to count the rules, not just the action. If you want to put those numbers on them I think you should also add those numbers in the comments of the source code.

After modifying the grammar, you may want to remember the reader to recompile the parse java file. By the way, when doing this I got the following error:
Note: ./ExprParser.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
I did as aked and it showed me the following warning:
ExprParser.java:155: warning: [unchecked] unchecked call to put(K,V) as a member of the raw type java.util.HashMap
memory.put(ID2.getText(), new Integer(expr3));
^
1 warning
I’m using javac 1.5.0_06 on Mac OS X 10.4.8. After a quick Google search (I do not know much about recent additions to Java), it’s because of the generics. To be sure not to have this error we have to use either HashMap<Object,Object> or HashMap<String,Integer> instead of just HashMap. But it would not work anymore for those using Java 1.4. So you should maybe add a note that it is not something we have to worry about.

The command line examples say ‘java Main’ whereas it should be ‘java Test’.

“you could refactor of the rule as follows” should be “you could refactor the rule as follows”

“\\{p}?” should be “{p}?” and “\\{p}?=>” should be “{p}?=>”

It is written "by matching the first alternative of rule @ r a’‘. The ’@ r’ has no reason to be there…

“you can usually used attributes” should be “you can usually use attributes”

In the sentence “in a directory with the same name such as org/antlr/codegen/templates/Java or org/antlr/codegen/templates/Java”, shouldn’t the second path end with an other language name (C, Ruby…)?

11.4 should be before 11.3 as it describes the backtrack option used in 11.3 and it references the nested grammar ‘above’ that’s in fact in 11.2 (7 pages before)

The introduction of this chapter is a hard to read, and after reading it I hardly had an idea of what is syntactic predicate. I think it’s missing some example or diagram.

The link to download the tour/basic/input files is incorrect. The book states the link media.pragprog.com/titles/tpantlr/code/tour/basic/input, but the real URL is media.pragprog.com/titles/antlr/code/tour/basic/input (antlr instead tpantlr).

The link to download the tour/basic/Expr.g file is incorrect. The book states the link media.pragprog.com/titles/tpantlr/code/tour/basic/Expr.g, but the real URL is media.pragprog.com/titles/antlr/code/tour/basic/Expr.g (antlr instead tpantlr).

Some chapter links are shown as chp. e.g. on pages 45, 46, and 55.

The note in the right margin is not completely readable at the bottom

The name of the class holding the return values should be r_return instead of rule_return to be consistent with the implementation of the implementation of the rule that follows

To be consistent with the conventions given in the table on page 41, should the tree grammer not be in a file called Expr.gtp instead of Expr.g?

Should the "else if (

In the sidebar the bolded switch has no spaces around it.

It looks like that “INTatom” should be written like “INT atom”. BTW, all links are wrong, as they have within the URL “tpantlr” instead “antlr”.

I’ve tried to test the /tour/eval/Expr.g with antlr-3.0b6 and ANTLRWorks 1.0b8. It seems, that the rule “‘(’ expr ‘)’ {$value = $expr.value;}” is incorrectly translated into “value = $expr.value;” instead into “value = $expr();” in ExprParser.java.

‘In a general purpose language like Java, to allow f( ) to access x,’ should read ‘In a general purpose language like Java, to allow h( ) to access x,’ as f() already has access to x.

Several of the attribute descriptions are incorrect:
“$x[–1 ::y]” should be “$x[–1]::y”, “$x[-i ::y]” should be “$x[-i]::y”, “$x[i ::y]” should be “$x[i]::y”, and “$x[0 ::y]” should be “$x[0]::y”.

“The following subsections describe it illustrate the operations I found.” should probably be something like “The following subsections describe the operations I found.”

Links to code do not work.

The variableDefinitions tree grammar is missing an ‘|’ character in its definition, as in

variableDefinition
: ^(VARIABLE ID modifier* type expression?)
| ^(FIELD ID modifier* type expression?)
;

‘new CommonToken(INT, “12”)’ actually is not the right choice for creating a new arbitrary token, at least for compatibility - ANTLRWorks (1.0b8 at least) does not properly display ASTs with Tokens defined this way, one instead needs to use ‘adaptor.create(INT, “12”)’.

The sentence “Lexical rules all begin with an uppercase letter in ANTLR and
typically refer to character and string literals not tokens as parser rules
do” needs commas around “not tokens”.

Broken link to download “tour/basic/Test.java”

The paragraph ending with “For example, ANTLR translates rule return specifications such as:” is followed by a method named “expr”, even though the grammar example has a rule for “multExpr”.

Awkward sentence: “Top-down parsers naturally associate operators left-to-right so that the
operators above are matched correctly using a natural grammar, but
what about operators the right associative such as the exponentiation
operator?”

Need comma (possibly “, and”) after “class level” in “In this case, the grammar is loosely written because decl should only
recognize a code block at the class level not in the statement level”

Need comma after “vocabulary symbols” in "By

Need comma after “is a sentence” in “For example, a complete
Java class definition file is a sentence as is a data file full of comma-
separated values.”

First use of DSL should define the acronym (same is true of other acronyms seen throughout). Might even want to define first use in each chapter or part.

Last paragraph on page seems to end abruptly, and does not appear to continue on the next page. Perhaps there was something like “following table” at the end?

Need comma after “following page” in “Figure 3.2, on the following page summarizes the tree
operators and rewrite rules.”

The textual version of the AST has an error in the third child of the ‘for’ node, as it represents ‘i=0’ and not ‘i=i+1’.
Similarly on page 136, in the image, the third child of the ‘for’ node is incorrect.

Chapter reference not filled in properly in ‘see chp.templateschp.templates.’

The arrows like <= and => are strangely pointed.

“javac …” is something I type. Its arrow is like =>.
“call foo;” is something I type, too, but its arrow is <=.

The whitespace rule should really be
WS: ( ’ ’ | ( ‘\\r’? ‘\
’ ) )+ { $channel = HIDDEN; } ;\t
In order to avoid errors in parsing when using windows.

Also the line
$ javac T.java Test.java
should be
$ javac TLexer.java TParser.java Test.java

Using ANTLR 3.0b6 (the latest as of 2/24/07), the command to compile and run Test should be:

javac TLexer.java TParser.java Test.java

instead of:

javac T.java Test.java

The provided grammar only allows ‘\
’ for WS, which fails on Windows machines. Added ‘\\r’ seemed to help.

“Adding a stack to a state machine turns it into a pushdown machine (pushdown automata).” It should read “automaton”. “Automata” is the plural form, but there’s only one machine.

Nitpck about the footnote: it should read “to err is human …”

The exception handler in the generated Java code is
catch (RecognitionException re) {
// error handling
}

But the grammar contains this
catch[RecognitionException re] {
System.err.println(“error”);
}

Shouldn’t the generated handler be
catch (RecognitionException re) {
System.err.println(“error”);
}

The text "Figure 3.2, on the next page shows the parse tree for

TestCMinus.java:

after the lines:

// Walk resulting tree; create treenode stream first
CommonTreeNodeStream nodes = new CommonTreeNodeStream(t);

the next line is missing:

nodes.setTokenStream(tokens);

(Without this call, a null pointer exception occurs. )

"…

“As an optimization, ANTLR collapses rules and subrules, whose alternatives are single token references without actions, into token sets as demonstrated in the following rule:
type: ‘int’ | ‘float’ | ‘void’ ;”

I’d like to see an unoptimized version of that for comparison.

“T Parser or tree parser. Match token T at the current input position. In a lexer, invoke lexer rule T. Tokens always begin with a capital letter.
T Lexer. Invoke lexer rule T.”

The parts between the * is mentioned twice. I suppose, that the part at the first position is to be deleted.

“For example, in the following Python code, the if statement and the method call to g( ) our at the same…”

In the above sentence, ‘our’ should be ‘are’.

Towards the bottom of the page:
“outer level whereas the the print statement and method call to”

‘the’ is incorrectly repeated as ‘the the’.

“Theafteraction is new and theinitaction was simply a code block in v2.” There are spaces missing for the bolded words ‘after’ and ‘init’.

It seems that the example of a StringTemplate group misses the opening <<-Brackets.

As my Java knowledge is limited, I’m not entirely sure about that one, but I believe that the catch-example with the rethrowing of the exception destroys the original stack trace. That has to be avoided.

Small grammatical error.
The first sentence after the invalid @ character example is missing the word ‘in’: “The recognizer found two errors [in] this case.”

At the top of this page it reads “The list the follows summarizes the special symbols and expressions…”. Perhaps it should read “The following list summarises…”

The ls displays Test.java already in the directory but the instruction to download it to there does not appear until the bottom of the page.

I suspect, that instead {System.out.println(“var”$id.text“:”$t.text“;”);} the action should be {System.out.println(“var”$id.text“, ”$t.text“;”);}.

“Because Java allows you to define variables in-line as opposed to the start of a code block, you can define variables inside of any action. The variables will be visible for the immediately surrounding alternative, but not other alternatives and not surrounding alternatives if the action is within a subrule.”

I’d appreciate some examples which show when which condition applies.

needs parser.program() instead of parser.prog()

“…lexer rules that refer to other lexer rules can access those portions of the overall token matched by the other rules and returned as implicit tokens.” Isn’t the word “be” missing in front of “returned”?

In the discussion of rule parameters, the text says that “rule declarator defines a parameter attribute called type that is available to actions as $type”, but the code immediately following has the rule declarator[String typeText] and references it as $typeText

“In the future, expect ANTLR to allow initialization code withinscopedefinitions like constructors.” The bolded scope has no spaces around it.

In the first paragraph…

to methods => two methods

And I wonder which is correct, in the running text, or in the code example:

displayRecognitionError() => getErrorMessage()

“In v2, there was aparaphraseoption that automated this”. The bolded word “paraphrase” has no spaces around it. Possibly a dot at the end of sentence missing.

“For example, clearly, the twoswitchstatements could be collapsed into a single one.” The bolded switch has no spaces around it.

“Template literals are either single-line strings in double quotes, ”…“. For multi-line strings use double angle-brackets, <<…>>,as shown in the highlighted section in the following grammar.” When I read the word “either”, I expect it to be followed by “or”. Regardless how the sentences are rewritten, the current version disturbs the flow.

"The first symbol after the

The first code sample (ANTLR 1.0 / PCCTS) has a missing closing curly brace for the “if” block:

expr();
} // ADDED CLOSING BRACE HERE
else if ( LA (1)…

The second code sample (ANTLR 3)is missing break statements after the two cases in the switch.

switch (alt) {
case 1 : // match alternative 1
match(ID);
match(EQUALS);
expr();
break; // ADDED BREAK HERE
case 2 : // match alternative 1
match(ID);
match(COLON);
stat();
break; // ADDED BREAK HERE
default : <>;
}

code comment is incorrect. It reads:

case 2 : // match alternative 1

but should be

case 2 : // match alternative 2

“the input to the translator is a series
expressions and assignments” should read “series of expressions”? And “can translate the text-based
bytecode assembly program to binary a Java .class file” should read “to a binary Java .class file”?

The following grammer snippet generates 2 semicolons for a ‘call()’. This is due to the string template being “; call(\\”\\“)”, where e is already a complete expression, with a semicolon.

expr: ID
| INT
| ^(CALL ID expr*)
-> {$stat::isAssign}? template(id={$ID.text},e={$text})
“eval(\\”\\“,)”
-> template(id={$ID.text},e={$text})
“; call(\\”\\“)”
;

(I even found it hard to type.)

— not ).

Sidebar text is not entirely readable due its position at the bottom.

Last paragraph “an existing grammars” should be “an existing grammar” or “existing grammars”.

Templates: You will see the full syntax…
Do not know how to write a simple test rig
to test the grammar with inline templates.
Example on the pages 213-214 does not help much.
—Wlodek Bzyl

“The general structure of a rule looks like the following: …”

The <> portion of a rule definition is listed here, but is never explained in the text. In particular, a description of the syntax and possibly and example would be nice. It would seem to fit with the section titled “Rule Exception Handling” which begins on page 101. Although the text on that page refers to a later chapter for more details about error handling, this clause is still not described in that later chapter.

The right-hand side annotation should use ‘omit’ and not ‘emit’.

The rule: “expr : ID
|INT
|ID ‘(’ (expr (‘,’ expr) )? ’)’ -> ^(CALL ID expr )
| ‘(’ expr ‘)’ -> expr
;”

Isn’t the rewrite “-> ^(CALL ID expr* )” more exact in this form: “-> ^(CALL ID expr+ )”?

First sentence in sidebar: “recognized” should be “recognize”.

Section 2.2, line 7: “Human” should be “human”.

missing `+’ in System.out.println

Second paragraph: ANTLRWorks will not generate code in the same directory as the grammar file by default. The output path can be configured in the preferences menu and by default it’s set to /tmp/antlrworks. At least in version 1.0b9 on Mac OS X.

The sixth footnote “cyclic DFA allow states to transition to previously-visited states” should have an Uppercase first word and a dot at the end.

“Reconsider the non-LL(k)class or interface”: Before the bolded class there is no space.

In the following sentence:

"The first symbol after the

The tesxt in the sidebar is missing spaces.

“Each alternative has a lookahead language and if a lookahead languages are disjoint for a decision then the decision is LL (*).” The “a” in front of “lookahead languages are disjoint” should be deleted.

The example rule

“primary
: ID ‘(’ exprList ‘)’ // ctor-style type cast or method call
-> ^(TYPECAST_OR_CALL exprList)
| ID
| INT
…
;”

may have an incomplete rewrite, where the ID token is missing.

Like on page 302 the example rule is missing on the second rewrite the ID token.

Looking at the code sample “void f(int ((*p))(float)){ body }”, I believe that “body” should be replaced by an italic “body”.

“This example merely demonstratew how syntactic predicates resolve true ambiguities by imposing order on alternatives.” “demonstratew” instead “demonstrates”.

“Both INT and ID syntactically predict the second alternative of rule a.” I don’t understand how this is possible with these rules:

“a : b
| {p2}? ID
;
b : {p1}? ID // only evaluate upon ID not INT
| INT
;”

INT can’t be identically lexically to INT, can it? Even taking into account that p1 is hoisted from rule b into rule a it would make only sense, if the sentence mentioned alternative one and not two.

"Java

"ANTLR resolves the ambiguity by forcing

second paragraph last sentence

“the parser reports and error” should be “the parser reports an error”

This part of rule example contains the reference to the variable y, even so this variable is no longer defined.

“:(b[y] ‘.’) => b[y] ‘.’
| b[y] ‘:’
;”

first paragraph under lexical rules second sentence

“lexer rules are always token names and began with” changes tense and should be “lexer rules are always token names and begin with”

second paragraph third sentence “Character positions ared indexed” should be “Character positions are indexed”

first sentence under section “Emitting More Than One Token Per Lexer Rule”

“force the lexer to emit more token per rule” should be “force the lexer to emit more tokens per rule”

second sentence under section “Tree Matching Rules”

“of the input stream but also encode the grammatical structure applied to that input string” should both references not be to the input stream?

section “output” second sentence

“Only available for combined, parser, and parser grammars” should be “Only available for combined, lexer, and parser grammars”?

first sentence “is a gated semantic predicates” should be “is a gated semantic predicate”

the second last sentence of the first paragraph is awkwardly worded “Always think of subtrees as operations with operands that tell the computer precisely what to do just like with the expression AST does above”. Perhaps removing the “does” or changing the “with” to “what” would make it more easily readable?

Though the first grammar “file : .+” is in line with the definition on the previous page (“[…] the structure is simply ‘one or more characters’”), a file can indeed be empty. Actually, both example programs support this (as does the second grammar!), so strictly speaking, the above grammar should read “file : .*” and the above definition should read “[…] zero or more characters”.

Though I didn’t test it, the “if (stripCurlies) { […] }” block looks wrong—if “getText()” is supposed to return the code block with surrounding curly braces, you not only want to get rid the opening one by means of “substring(1,…)”, but you also need to correct the string length in order to drop the closing one as follows:
“setText(getText().substring(1, getText().length() - 2));”

Sigh… the example is wrong, but so was my first correction; you can only get rid of the last curly brace by modifying the end index as follows:
“setText(getText().substring(1, getText().length() - 1));”
[note so self: Eclipse provides a scrapbook facility, so use it!]

Apple guidelines say that you should refer to the operating system as “Mac OS X”, not “OS X” as here.

Apple guidelines say that you should refer to the operating system as “Mac OS X”, not “OS X” as here.

Says “root note” instead of “root node”.

first sentence of the second paragraph
“number of techniques describe by the previous sections”
should be
“number of techniques described by the previous sections”

In paragraph two “context-tree” should be “context-free”.

In section 10.5, “reports the errors usual” should read “reports the errors as usual”.

second paragraph second sentence “language and if a lookahead languages are disjoint” should be
“language and if a lookahead language is disjoint”

“which causes the unreachable alternative error” should really be “which causes the unreachable alternative warning”.

first paragraph third last sentence
“The gated semantic predicates dictates”
should be “The gated semantic predicates dictate”

first paragraph first sentence “disambiguating semantic predicates” should be “disambiguating semantic predicate”

backtrackign -> backtracking.
in “but nested backtrackign can be the source of some
confusion”

Using antlr-3.0b7, “java org.antlr.Tool infloop.g” prints:
—-
ANTLR Parser Generator Version 3.0b7 (April 12, 2007) 1989-2007
warning(200): infloop.g:8:5: Decision can match input such as “‘int’” using mult
iple alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
—-
The result is different from that in this page.
Also, with antlrworks-1.0b10, the DFA for slist is different from that in this page.

tree built for “a=3;”
-> tree built for “a=3” followed by NEWLINE

1. or something like that.
2. Expr.g expects NEWLINE, not semicolon, as delimiters of stat.

third paragraph first sentence ’containing a Token

see @scope.

1) it’s a typo should be scope..

2) I’m not sure why init and after have @
and scope doesn’t have.
It looks inconsistent from the first sight…

rulecatch example uses RecognitionException e.
That looks nice, but doesn’t work, because “e is already defined”.

The simple tree based interpreter referenced in [3] has some errors.

ANTLR Parser Generator Version 3.0b7 (April 12, 2007) 1989-2007
error(100): Eval.g:73:2: syntax error: antlr: Eval.g:73:2: unexpected token: finally
error(100): Eval.g:0:0: syntax error: define: :73:10: expecting ID, found ’
scopes.pop(); // remove arg scope
’
error(10): internal error: Eval.g : java.lang.ClassCastException: antlr.ASTNULLType
org.antlr.tool.DefineGrammarItemsWalker.block(DefineGrammarItemsWalker.java:1070)
org.antlr.tool.DefineGrammarItemsWalker.rule(DefineGrammarItemsWalker.java:836)
org.antlr.tool.DefineGrammarItemsWalker.rules(DefineGrammarItemsWalker.java:488)
org.antlr.tool.DefineGrammarItemsWalker.grammarSpec(DefineGrammarItemsWalker.java:333)
org.antlr.tool.DefineGrammarItemsWalker.grammar(DefineGrammarItemsWalker.java:174)
org.antlr.tool.Grammar.setGrammarContent(Grammar.java:551)
org.antlr.tool.Grammar.(Grammar.java:448)
org.antlr.Tool.getGrammar(Tool.java:331)
org.antlr.Tool.process(Tool.java:267)
org.antlr.Tool.main(Tool.java:70)

In the source code, line 6, a tab is missing before // int i

That listing again, line 7: the third term in the for loop (i=i+1) is translated to (= i 0). Shouldn’t that be (= i (+ i 1)), or something like? The same mistake is shown in the tree structure on the following page.

2nd bullet, end of 2nd line, “World”: should this be capitalized?

In the “Finite state machines” sidebar, at the end of the first para, I find the discussion of LL (*) vs. LL (k) confusing as the terms are unfamiliar at this point and I don’t think that the last sentence (“ANTLR’s LL (*) parsing…k+1 states.”) is necessary anyway.

The 2nd para after the code says “All of the FOLLOW_multExpr_in_expr160 variable and…” I don’t understand what “FOLLOW_multExpr_in_expr160” is unless it’s some internal macro that didn’t get translated properly?

I started to get a bit lost with my eyes glazing over once I started reading the section entitled “Categorizing Recognizers”. At this point I was eager to start applying the knowledge gained so far, but it’d be 8 pages before I get to the next chapter. Perhaps these 8 pages could be moved later? It doesn’t seem necessary to understand LL vs. LR or LL (k) in order to try out the examples in Chapter 3.

Assuming the examples are correct, several of the specified operations have typos, as follows:
1) $x[–1 ::y] should be $x[–1]::y
2) $x[-i ::y] should be $x[-i]::y
3) $x[i] ::y] should be $x[i]::y
4) $x[0 ::y] should be $x[0]::y

Missing full stop in side comment: “Recursive rules in the lexer are useful for matching things like nested comments(. C)ontrast this…”

The last sentence of the second paragraph has a typo:

“…alternative simply means don’t creat a tree …”

should be

“…alternative simply means don’t create a tree …”

Map symbols = new HashMap();

Should probably be:
HashMap symbols = new HashMap();

In the right sidebar note, “In v2, there was aparaphraseoption that automated this” there are missing spaces around paraphrase.

Should be “In v2, there was a paraphrase option that automated this”

In the PDF version of the document, and I’m guessing the print version, it seems incorrect to insert Table 10.1 right in the middle of a code listing. P.g starts on page 246, is interrupted on page 247 by Table 10.1, and then continues on 248.

In the after code for func, it prints out CScope::name, but the init code doesn’t set CScope::name. Prog and block do, but not func.

Section ‘Building ASTs with a Grammar’
Para #2
Sentence #2
Phrase ‘As you we along’ at the beginning

Figure 4.2
In r^ description, last sentence.
“Rule r should must”

The last sentence in the book is blocked by the dark blue bar (the one that contains the phrase: Prepared exclusively for….)

I am not sure if this is an error in the book or the website but the link in footnote 2 to www.antlr.org/v3 doesn’t seem to work.

The beginning of paragraph 3 starts with “A grammar describe”, which should be “A grammar describes”.

The last line is invisible, also on some other pages.

In many cases (page 83 is just an example) the “customization” bar at the bottom of the page covers the last line of text.

Fourth paragraph, third line “symbols or its children.” should be “symbols are its children.”.

Second paragraph (after sample code), first line says "to tokens +, , (, and )." It should read "to tokens +,, *, (, and ). Also, adding single quote marks around the symbols would make it read better.

I can’t read the bottom line of page 41 “upon reaching the end of that submachine.” as it is obsured by the “Report erratum” text with a blue background.

The url referenced in footnote #6 can not be found.

The example at the top of the page left out the TRUE subrule for the IF. It also left off the indicator that stat could appear multiple times. It should probably read something like:
… ‘(’ expr ‘)’ (stat)? ( ’else’ stat )? …

Last paragraph, first line should read:
Lexer rules can force the lexer to emit more than one token…

On this page a section is started on “Tree Matching Rules” with no previous discussion of tree creation rules. A page or two on tree creation before tree matching would help things flow smoother and keep things in context.

section ‘Arithmetic Expression Grammars’
first paragraph
first sentence, first row
ENBF should be EBNF

The last line is “invisible”, also on some other pages.
Please resend PDF.

“loath” should be “loathe” (the verb) in the sentence “It is because of this ambiguity and many others that language imple-
menters loath C.”

DOTTreeGenerator does not seem to exist in the CSharp runtime.

URLs in the footnote are reversed.

pageId=1760 is the parse tree page.

pageId=1763 is the heterogeneous tree page.

I think the first java method’s name should be multExpr() but not expr().

The footnote isn’t entirely readable, as the last sentence is hidden by the blue bottom.

The last sentence is hidden by the blue bottom.

The upper box refers to “slist”, which isn’t used in the code samples. Either “slist” is “prog”, or more likely, “stat”.

On page 210, in the templates/generator/1pass/input.j example, the constructor invocation is incorrect. It compiles but you can’t instantiate the assembled classfile as an object, the contructor line should look like this and include the keyword “” which is omitted:

.method public ()V

And same thing when calling on Object.super() constructor, just below:

invokenonvirtual java/lang/Object/()V

The book example actually works since it doesn’t call for actually creating an object instance since generated test example is in a static method, but I wanted to instantiate the assembled class.

On page 181 of the PDF and page 168 of the dead-tree version, we have:

variableDefinition
: modifiers type ID (‘=’ expression)? ‘;’
-> {inMethod}? ^(VARIABLE ID modifier* type expression?)
-> ^(FIELD ID modifier* type expression?)
;

…where “modifiers” should be changed to “modifier*” (or vice-versa).

“Lexer rules can force the lexer to emit more token” should be “Lexer rules can force the lexer to emit more than one token”. Also, you probably should move the sentence that comes before the “Emitting More Than One Token per Lexer Rule” down after the header.

In the Literals section, the “one-character string containing the single quote character” is missing the closing single quote. I.e. it is shown as ‘\\" instead of ’\\"’

On the second line of the comments in Expr.g, there are extra spaces in the $stat.tree.toStringTree() call right before toStringTree.

After the testing the expression recognizer, the text says “ANTLR doesn’t emit any output because there are no actions and grammar”. There is definitely a grammar… this must mean "there are no actions in the grammar.

“There are two tokens in this case:” is followed by a list of three tokens with types 4, 5, and 6. One is a literal ‘+’, but isn’t that still a token?

The first sentence about the ASTLabelType option contains “all node variables pointers”, which doesn’t look right. Shouldn’t it be “all node variable pointers”? Or maybe it’s intended to be possessive: “all node variables’ pointers”.

The description of Actions says “to get the close curly character, escape it with a backslash”. But there are many examples in the book that use Java or C syntax with brace-enclosed blocks of code (within a brace-enclosed grammar action), and the } characters aren’t escaped. What is the whole story? I suspect that matching braces are ignored without escaping, but how are those inner braces found? (How does ANTLR tokenize to find the braces, when it doesn’t know the target language and so doesn’t know what’s a comment or a string literal?) This should really be explained more fully, so that users know when they really need to escape } characters… which might depend on their target language.

“Any rule invoked from the methodDefinition rule can use the boolean instance variable inMethod to test…”. I don’t think this is explained properly, because any rule, not just those invoked from methodDefinition, can test inMethod (I think). Tests in rules invoked from methodDefinition will find inMethod=true, while other tests will find it false, but they can all test it.

The rewrite rule for forStat doesn’t reference the statement list slist. It should probably do that, right?

The first paragraph of Chap 9 contains “from an internal data structures”. The cardinalities are mismatched :-) It should say “an internal data structure” or “internal data structures”.

The last paragraph of section 9.1 has a subject/verb mismatch: it should say “Isolating […] makes sense” instead of “make sense”.

The lead-in to the example says “override two methods from BaseRecognizer, displayRecognitionError( ) and getTokenErrorDisplay( )”. However, it looks like getErrorMessage( ) and getTokenErrorDisplay( ) are overridden in the example. Should “displayRecognitionError” be changed to “getErrorMessage” in the lead-in?

There is no space separating the second sentence in section 10.3 from the first sentence. It says “… grammar.For …”.

The comment in the code says “catch-clauses get replace with this action”. That should be “replaced”.

The paragraph beginning “The beauty of this solution is that a syntactic predicates handles …” contains a typo. It should say “a syntactic predicate” (singular) OR remove “a” and change “handles” to “handle”.

At the bottom of the page when explaining the big DFA that does the lookahead for the embedded subrule of interfaceDef, the text says that DFA scans until the semicolon or left curly. It is actually left paren (not curly - the paren is the start of the method’s arg list, which disambiguates), or ASSIGN or semicolon (variable def with or without init).

This page uses “DFA” for the plural of “DFA” several times (I count 5 times). While I realize that the plural of “automaton” is “automata” and not “automatons”, many places elsewhere in the book (including the next page) use “DFAs” for the plural of “DFA”, and I think that that reads better. Anyway, one consistent usage should be chosen. Note that the last plural “DFA” - in footnote 6 - mixes up the plural/singular verb form; it should say “allow” because the subject is plural, as in “Cyclic DFAs allow states to …”. Or it could use “A cyclic DFA allows …” of course.

The second paragraph begins with “lThis” which should just be “This”.

The third paragraph say “unless the uncovered alternatives is last”. That should be singular “alternative”.

Last sentence in Fig 4.2 reads “Rule r should must a single node, not a subtree.” Maybe it should be “should return a single node, …” ?

The reader is told to type an end-of-file character but the footnote explaining how to type the end-of-file character doesn’t appear till page 66 (in the PDF).

Quote from page 30: “Note that you must type the
end-of-file character to ter minate reading from standard input”

middle of page: You could say that the file has no structure or that the structure is simply “one or more characters.”

Should be “zero or more characters.”

The comment in the code block of the following page is correct.

The code on Page 215 (Paper) for “expr” has a return value (returns [int value]) which doesn’t seem to be necessary, maybe it’s a copy-paste error from the example on page 65 (Paper)?

The instruction to compile Test.java, ExprParser.java and ExprLexer.java fails when using the code as distributed from this site, thus:

Test.java:28: cannot find symbol
symbol : class Eval
location: class Test
Eval walker = new Eval(nodes); // create a tree parser
^
Test.java:28: cannot find symbol
symbol : class Eval
location: class Test
Eval walker = new Eval(nodes); // create a tree parser
^
2 errors

This is corrected by comment out the Walking Tree block in Test.java and compiling.

Test.java as distributed is appropriate once tree grammar has been written and compiled.

Perhaps insert appropriate Test.java code mid page 62?

In the upper half of the page, a source code sample is given. There is stated that a grammer such as:

multExpr returns [int value]

will lead to the Java Code:

public int expr() throws …

I assume this should be:

public int multExpr() throws …

The tree printing code throws a NullPointerException woth an empty expression line, buttoken NEWLINE is a valid expr.

prog :(stat {System.out.println($stat.tree==null?null

or something similar would fix this.

“MATCh” in the first chunk of code should be “MATCH”.

The example of a program that consumes all characters:

file : .+ ;

throws a “Lexer is null” exception when run in the ANTLRworks interpreter. A possible alternative is to express it using a lexer line, i.e. capitalize “file”, as in:

FILE : .+ ;

which the interpreter does accept.

“backtracking=true” should read “backtrack=true”

“backtracking=true” (twice) should read “backtrack=true”

While this is not germane to the ANTLR point discussed there, List<List> is not valid Java and should probably read List<List>.

The last sentence on the page is not visible due to the blue border.

Minor typo on 2nd paragraph, first line:

“ambiguous sentence or phase”

I think that should be “phrase”.

In the generated code for a(), the “System.out.println(”exiting“);” line should actually be “System.out.println(”after matching rule; before finally“);” per the definition of @after on the preceeding page.

All three lines of the export statement should be bold; as is,
the second and third line look like output.

This is a minor point, but in the paragraph discussing $ruleRef, I think the references to “$rulename” and “rulename” should be updated to “$ruleRef” instead, for consistency.

In the anecdote box at the top of the page:

“”"
“That ’;’ came as a complete surprise to me at this point your program.”
“”"

Should be “this point OF your program”.

“This is a good way to distinguish between local variables and fields of a class definition, for example.”

I’d say it is a way, but not really a good way to do that. Just five pages earlier a better way, using a scope, is explained. Also, the @after block will be skipped if an exception is thrown so in that case inMethod will be corrupted. inMethod should be set to false in the finally block.

“When generating templates, on the other hand, rewrite rules specify the template to create and a set of argument…”

should be

“When generating templates, on the other hand, rewrite rules specify the template to create a set of argument…”

and BTW I would advice you to rewrite that long sentence into shorter ones, makes it more readable (and less error prone :-).

“..on page 44 and augment it to build a suitable AST. As you we along..”

should be

“..on page 44 and augment it to build a suitable AST. As we move along..”

The example for the second alternative of the method rule is wrong. The rule args requires that there is at least one argument to a method. A correct example for the second alternative of the method rule would be: int f(int z) {…}

Expr.g not produced, ExprLexer.java is produced, using
version 3 of Antlr

In ‘list of generated files’

Test.java
Shown as calling parser.prog()
should be
parser.stat();

which is the first rule in the grammar.

$ java org.antlr.Tool Eval.g
ANTLR Parser Generator Version 3.0 1989-2007

Running with 3.1 reports a missing file
ANTLR Parser Generator Version 3.1 (August 12, 2008) 1989-2008
File EvalLexer.java is missing

and does not create the listed files
Eval.java and Eval.tokens.

No mention of the .token file from Expr.g?

Again the provided code is a mess
tree grammar Eval; options { tokenVocab=Expr; ASTLabelType=CommonTree; } // START:members header { import java.util.HashMap; } members { / Map variable name to Integer object holding value */ HashMap memory = new HashMap(); } // END:members // START:stat prog: stat+ ; stat: expr {System.out.println($expr.value);} | ^{(’=’ ID expr) {memory.put($ID.text, new Integer($expr.value));} ; // END:stat // START:expr expr returns [int value] :}(’’ a=expr b=expr) {$value = a+b;} | ^{(’-’ a=expr b=expr) {$value = a-b;} |}(’*’ a=expr b=expr) {$value = a*b;} | ID { Integer v = (Integer)memory.get($ID.text); if ( v!=null ) $value = v.intValue(); else System.err.println(“undefined variable ”$ID.text); } | INT {$value = Integer.parseInt($INT.text);} ; // END:expr

I noted that ‘expr.g’ was well laid out.

Near top, says
For rule r, ANTLR generates the following code where the highlighted
executable lines derive from rule r’s parameters and return values:

There appear to be no ‘highlighted executable lines’

Minor typo. Last sentence on page: “When you have ambiguities in other non-determinisms in your grammar…”. I think this should read: “When you have ambiguities and other non-determinisms in your grammar…”?

In the end of first paragraph: “assuming type matches matches ‘int’”. I think it should be just “assuming type matches ‘int’”.

In the end of second paragraph: “can also be a statements”. I think it should be “statement” (singular)

In first paragraph: “… this means syntactic predicates let you use write grammars that ANTLR …”

Maybe the sentence should be: “… this means syntactic predicates let you write grammars that ANTLR …”

Second paragraph, second paragraph after the code snippet: “It reflects a situation where you want to treat identifiers s and sx differently depending on whether they are followed by an integer”

Paragraph 3, sentence 1 - “A grammar describe …” should be “A grammar describes …”.

Paragraph 2 and 1st method name in following example code disagree. Should they both be “displayRecognitionError()” or “getErrorMessage(…)”?

Paragraph 2 (following 1st diagram): The paragraph begins with a spurious “1”.

In Figure 4.1, make the “.” bolder. I find I repeatedly miss it, when looking at this table. Also, the utility of this entry as a reference would be improved by refering people to the discussion beginning at the bottom of p.85 on “.*” and “.+”.

Look at this sentence from that page:
“Sentences are ambiguous if at least one of its phrases
is ambiguous.”
I think it should be “A sentence is ambiguous if at least one of its phrases is ambiguous”

The link cited in footnote 12 is no longer valid.

About half way down the page, the phrase “but where the semicolon can also be a statements:”, the last word should be singular (“statement”).

At end of page, and flowing onto the next, “choose which rule to match upon saying a single-digit such as 4.” has two problems: “saying” should be “seeing,” and the hyphen in “single-digit” shouldn’t be there.

Building ASTs with a Grammar, second paragraph, second sentence:

As you we along, …

The example grammar on p. 264 requires at least one token A to occur. Thus, the rewrite on the facing page should be

a : A a
| A
;

Or, using EBNF instead of tail recursion

a : A+ ;

The WS rule in tour/basic/Expr.g includes ‘\
’|‘\\r’
With input like:

1+2\

\

\

4+5

the WS rule eats the newlines between 2 and 4, yielding “missing NEWLINE at u’4”

(Footnote) “rule references within alternative” needs more letters. “rule references within an alternative” or “rule references within alternatives”.

(Top of page) To be consistent with graph, should read ^{(CLASS T}(VARDEF …

(Last paragraph) The last of the three clauses in the for statement is referred to as ‘the initializer’. If this is standard Java nomenclature, OK, but I think of the first clause as the initializer and the third as a updater or re-initializer. On first reading, it seemed like this paragraph referred to the first clause, which didn’t make sense when the paragraph said that “the initializer expression is last.”

(first sentence) “matches” appears twice, once at end of line and once at beginning.

Page 60 states “To specify a tree structure, simple indicate which tokens should be considered operators (subtree roots) and which tokens should be excluded from the tree. Use the ^ and ! token references, respectively.”

There is then a code example which shows:
| ‘(’! expr ‘)’!

Shortly after, “We only need to add AST operators to tokens +, -, *, (, and ).”

This appears to be a contradiction. The code example indicates that “(” and “)” should be excluded from the tree, yet we are told that we need to add AST operators to parentheses.

Am I misreading, or is this an erratum?

Sentence has two verbs. Should probably be either “use” or “write” ;)
“In practice, this means syntactic predicates let you use write grammars
that ANTLR would otherwise reject. ”

Sample code seems not to properly match type with constructor in:
MyParser parser = new MParser(tokens,symtab);
Probably typo in either MyParser or MParser.

Figure 3.2: newline symbol “\
” is missing in the second child of rule “stat”

export CLASSPATH=“$CLASSPATH: ”foo

“foo” is extraneous garbage

the last part of this page acan’t be seen

After listing the Test rig, it would help to show it running with some test input data. Also, the ByteCode.stg listing shown on earlier pages does not include the full listing. I downloaded the code to obtain this. Perhaps a note about this would be helpful.

The description of r^ in Figure 4.2, the last sentence is like this: Rule r should must a single node, not a subtree.

My question is: can ‘should must’ be use together?

the description of ^ in figure 7.1 is not acurate:

Next nonsuffixed element’s node or subtree becomes the first child of this root node…

It’s not necessay to be the FIRST child, like this case:

expr : INT +^ FLOAT;

the INT is the first child, not FLOAT, the tree is like this: + DOWN INT FLOAT UP

Strange ‘foo’ at end of this:

the bash shell, you can do the following:
$ export CLASSPATH=“$CLASSPATH:/usr/local/antlr-3.0/lib/antlr-3.0.jar:\\ /usr/local/antlr-3.0/lib/stringtemplate-3.0.jar:\\ /usr/local/antlr-3.0/lib/antlr-2.7.7.jar”foo

wrt Section 1.3, the directions are not even close to correct!
Nor is the website clear. I would rather be reading the book and following the examples than debugging messy configuration stuff. I should not need to “check that you’re running the same version of software used in the book”. Rather, the book should be written to handle updates to the software, which are certain to occur. I expect better from “The Pragmatic Bookshelf”.

Regards
Arthur

After “We are going to modify the new nib so that it becomes the user interface
for our MovieEditorViewController. The first step is to change the class of
the File’s Owner. Select this object, and open the Identity inspector with
D-4. Change the Class field to MovieEditorViewController. The inspector
should update with all the outlets and actions that we added to the
header and look like this:”, the screen shot should show “MovieEditorViewController” for the class of File’s Owner, instead it shows “MovieViewController”.

In the comments prior to the expr rule, it says that you multiply the results of the two multExpr values when, in fact, you either add or subtract the two multExpr values based on the operator token.

In the fourth paragraph, second sentence, it looks like the second appearance of the word “alternatives” should be removed.

It currently reads, “Alternatives that are not mutually ambiguous, even in the same block alternatives, do not need syntactic predicates.”.

It should read, “Alternatives that are not mutually ambiguous, even in the same block, do not need syntactic predicates.”

In the second paragraph, first sentence, the word “implementation” should be plural. It shoud read, “The global dynamic scope mechanism works well for many simple symbol table implementations such as this, …”.

MyParser parser = new MParser(tokens,symtab);
should be
MyParser parser = new MyParser(tokens,symtab);

For grammar file T.g, if you’re creating this manually, make sure there is a blank line at the end of the file. It seems that ending with a single-line comment without a newline does not parse correctly.

This is the error I encountered:
C:\\WORK\\lib\\antlr\\test\\Introduction>java org.antlr.Tool T.g
error(10): internal error: Exception T.g:5:64: unexpected char: ‘e’@org.antlr.grammar.v2.ANTLRLexer.nextToken(ANTLRLexer.java:346): unexpected stream error from parsing T.g

error(150): grammar file T.g has no rules
error(100): T.g:0:0: syntax error: assign.types: :0:0: unexpected end of subtree
error(100): T.g:0:0: syntax error: define: :0:0: unexpected end of subtree

Figure 4.2: r^ (last sentence) Rule r should must a single node, not a subtree.

footnote 3 contains a broken link. (I can’t post the link or the link to the correct page because pragprog system think I’m spamming if I do so). Please update the link or provide a working http redirect on the original link

The link in the footnote is broken(page not found).

r @init { Token myFirstToken = $start; // do me first }

should this not be:
…
Token myFirstToken = $r.start;

Last paragraph, first sentence:
The problem is that the looping subrule in declarator_specifiers…
should be:
The problem is that the looping subrule in declaration_specifiers…

When running the Expr test for ‘trees’ the book text says:

BEGIN QUOTE

First ask ANTLR to translate Expr.g to Java code and compile as you did for the previous solution:

$ java org.antlr.Tool Expr.g
ANTLR Parser Generator Version 3.0 1989-2007
$ javac Test.java ExprParser.java ExprLexer.java

END QUOTE

These instructions are confusing because the “Test.java” file included in the book’s code (tpantlr-code.tgz) is the modified Test.java rig used on page 66.

This is confusing since the new ExprParser.java and ExprLexer.java files are generated into the same directory as the Eval version of the test rig.

My suggestion is that the book should say, for the test rig on page 62, specifically something about using the Test.java test rig from the previous solution. Simply running what it given in the book (“$javac Test.java ExprParser.java ExprLexer.java”) results in a compile error.

“Generating templates behaves just like generating trees using rewrite rules.Each rule implicitly returns a template or tree, and the -> rewrite operator declares the output structure.”

Specifically, the full stop following “rules” should be followed by a space.

(Also, I look forward to ANTLR4, thanks!)