Language parsing

• Java SE Specifications

• PHP: List of Parser Tokens - Manual

• BNF for specific URL schemes RFC 1738 - A Gopher URL Format (see also RFC updates RFC 1738)

• GOLD Parsing System - Grammar Downloads

• Syntax (programming languages) — Wikipedia

Parsing and transpiling

• jamiebuilds/the-super-tiny-compiler: Possibly the smallest compiler ever

• Compiling Java and C# to SWF – blog.joa-ebert.com - Blog of Joa Ebert

• How to be* a compiler — make a compiler with JavaScript – Medium

• Easy Parsing with Parser Combinators

• Implementing a programming language in C - Part 1

• Parsing CSS in JavaScript / jQuery - Stack Overflow

• use source maps Compiling to JavaScript, and Debugging with Source Maps ★ Mozilla Hacks – the Web developer blog

• How to Writing a parser

• Writing Your Own Toy Compiler Using Flex, Bison and LLVM (gnuu.org)

• Free Compiler Construction Tools: Lexers, Parser Generators, Optimizers (thefreecountry.com)

• CmSc 315 Programming Languages

• Peachpie – The open-source PHP compiler to .NET

• niklasvh/php.js: PHP to JavaScript converter and VM written in JavaScript

• Crafting Interpreters - munificent/craftinginterpreters: Repository for the book "Crafting Interpreters"

• Building a Scanner – Chelsea Troy

• Building a Parser – Chelsea Troy

• Drawings of How Compilers Work – Chelsea Troy

• Stuff I Wrote About Programming Languages – journal.stuffwithstuff.com

• JSON Parser with JavaScript | Tan Li Hau - Write a parser with JavaScript | Hacker News

• esbuild/architecture.md at master · evanw/esbuild - esbuild bundler architecture

• Let's write a compiler

  • Let's write a compiler, part 1: Introduction, selecting a language, and doing some planning - Brian Robert Callahan

  • Let's write a compiler, part 2: A lexer - Brian Robert Callahan

  • Let's write a compiler, part 3: A parser - Brian Robert Callahan

Concepts

Long switch in a loop, can be used for state machine, but it's not adviced to use long switch for parsing, because it's often need multiple states (to restore previous ones). Use recursive descent parser instead.

• Parsing — Wikipedia

• Parse tree — Wikipedia

• Abstract syntax tree — Wikipedia

• Analyse syntaxique — Wikipédia

• Lexical analysis — Wikipedia

• Lint (software) — Wikipedia

• Parsing expression grammar — Wikipedia

• CSS parsing: performance tips & tricks

• postcss/architecture.md at master · postcss/postcss

Generators

• Comparison of parser generators — Wikipedia

• PEG.js – Parser Generator for JavaScript - see PEG.js: Parser generator for JavaScript

• re2c — re2c 0.16 documentation

• fastparse: Writing Fast Parsers Fast in Scala

• Jison - Bison in JavaScript / JavaScript parser generator. See https://github.com/zaach/jison

• A Parser Combinator library for C

• Build parsers in Java

• tree-sitter/tree-sitter: An incremental parsing system for programmings tools

• SAP/chevrotain: Parser Building Toolkit for JavaScript

Parser examples

Tokenizer + parser

• tree-sitter/tree-sitter: An incremental parsing system for programming tools - Create a grammar and generate a parser

• retextjs/retext: Natural language processor powered by plugins based on @unifiedjs (and @vfile, @syntax-tree) - Natural language parser (latin, english, etc.)

• shift-regexp-acceptor-js/index.js at es2016 · shapesecurity/shift-regexp-acceptor-js

• Esprima - ECMAScript parser written in ECMAScript. See https://github.com/jquery/esprima

• JSCSSP written by Daniel Glazman

  • therealglazou/jscssp: a fast and extensible CSS parser written in JavaScript

  • JSCSSP, a CSS parser in JavaScript

  • therealglazou/JSCSSPHX

  • https://dxr.mozilla.org/mozilla-central/source/layout/style/nsCSSParser.cpp

  • https://dxr.mozilla.org/mozilla-central/source/layout/style/nsCSSScanner.cpp

  • NV/CSSOM: CSS Object Model implemented in pure JavaScript. It's also a parser!

• Mascara JavaScript Compiler - see also News - Mascara - The JavaScript of the future, today

• Classes.Math Parser - JSFromHell.com: JavaScript Repository

• Parser API - Mozilla | MDN

• GLSL to HLSL transpiler code

• Parse, Analyse, Clean GLSL code in JS

• Jsmn is a world fastest JSON parser/tokenizer

• Very low footprint JSON parser written in portable ANSI C

• Super-fast Java JSON parser

• MBED3 port of Lightweight Embedded JSON Parser

• A fast and small JSON parser and writer for Java

• pvdz/tenko: An 100% spec compliant ES2020 JavaScript parser written in JS

• inikulin/parse5: HTML parsing/serialization toolset for Node.js. WHATWG HTML Living Standard (aka HTML5)-compliant.

• JamesNK/Newtonsoft.Json: Json.NET is a popular high-performance JSON framework for .NET

Use gotos, "This is called computed (or assigned) goto and is a GCC extension"

• https://github.com/quartzjer/js0n/blob/master/src/js0n.c

• https://gcc.gnu.org/onlinedocs/gcc/Labels-as-Values.html

• https://news.ycombinator.com/item?id=4387156

Syntax exemples:

foobar : "foo" | "bar";

// (4 * 2 * 11 + 2) - 5
// 29 + 2 * 3 - 99 - (5 + 5 + 2) / 100
expression : <product> (('+' | '-') <product>)*;
product : <value>   (('*' | '/')   <value>)*;
value : /[0-9]+/ | '(' <expression> ')';
maths : /^/ <expression> /$/;

// so c so c so c so c so c so c so c so c so c
// wow c so language such book
adjective : "wow" | "many" | "so" | "such";
noun      : "lisp" | "language" | "c" | "book" | "build";
phrase    : <adjective> <noun>;
doge      : /^/ <phrase>* /$/;

ident     : /[a-zA-Z_][a-zA-Z0-9_]*/ ;
number    : /[0-9]+/ ;
character : /'.'/ ;
string    : /\"(\\\\.|[^\"])*\"/ ;

factor    : '(' <lexp> ')'
          | <number>
          | <character>
          | <string>
          | <ident> '(' <lexp>? (',' <lexp>)* ')'
          | <ident> ;

term      : <factor> (('*' | '/' | '%') <factor>)* ;
lexp      : <term> (('+' | '-') <term>)* ;

stmt      : '{' <stmt>* '}'
          | \"while\" '(' <exp> ')' <stmt>
          | \"if\"    '(' <exp> ')' <stmt>
          | <ident> '=' <lexp> ';'
          | \"print\" '(' <lexp>? ')' ';'
          | \"return\" <lexp>? ';'
          | <ident> '(' <ident>? (',' <ident>)* ')' ';' ;

exp       : <lexp> '>' <lexp>
          | <lexp> '<' <lexp>
          | <lexp> \">=\" <lexp>
          | <lexp> \"<=\" <lexp>
          | <lexp> \"!=\" <lexp>
          | <lexp> \"==\" <lexp> ;

typeident : (\"int\" | \"char\") <ident> ;
decls     : (<typeident> ';')* ;
args      : <typeident>? (',' <typeident>)* ;
body      : '{' <decls> <stmt>* '}' ;
procedure : (\"int\" | \"char\") <ident> '(' <args> ')' <body> ;
main      : \"main\" '(' ')' <body> ;
includes  : (\"#include\" <string>)* ;
smallc    : /^/ <includes> <decls> <procedure>* <main> /$/ ;

node : '(' <node> ',' /foo/ ',' <node> ')' | <leaf>;
leaf : /bar/;
input : /^/ <node> /$/;


number  \"number\"  : /[0-9]+/ ;
symbol  \"symbol\"  : /[a-zA-Z0-9_+\\-*\\/\\\\=<>!&]+/ ;
string  \"string\"  : /\"(\\\\.|[^\"])*\"/ ;
comment             : /;[^\\r\\n]*/ ;
sexpr               : '(' <expr>* ')' ;
qexpr               : '{' <expr>* '}' ;
expr                : <number>  | <symbol> | <string>
                    | <comment> | <sexpr>  | <qexpr> ;
lispy               : /^/ <expr>* /$/ ;

// #line 10 "test"
number        : /[0-9]+/ ;
quoted_string : /\"(\\.|[^\"])*\"/ ;
linepragma    : <line> <number> <quoted_string>;
parser        : /^/ (<linepragma>)* /$/ ;

// [my_func]\n  echo (a b c)\n
qscript        : /^/ (<comment> | <resource>)* /$/ ;
	comment     : '#' /[^\\n]*/ ;
resource       : '[' (<rtype> <rname>) ']' <inner_block> ;
	rtype       : /[*]*/ ;
	rname       : <qstring> ;

inner_block    : (<comment> | <statement>)* ;
	statement   : <function> '(' (<comment> | <parameter> | <block>)* ')'  <seperator> ;
	function    : <qstring> ;
	parameter   : (<statement> | <literal>) ;
		literal  : (<number> | <qstring>) <seperator> ;
	block       : '{' <inner_block> '}' ;
	seperator   : ',' | \"\" ;

qstring        : (<complexstr> | <simplestr>) <qstring>* ;
	simplestr   : /[a-zA-Z0-9_!@#$%^&\\*_+\\-\\.=\\/<>]+/ ;
	complexstr  : (/\"[^\"]*\"/ | /'[^']*'/) ;

number         : (<float> | <int>) ;
	float       : /[-+]?[0-9]+\\.[0-9]+/ ;
	int         : /[-+]?[0-9]+/ ;

Syntax formats

• Backus-Naur (Backus–Naur) form:

• Augmented Backus-Naur form (Augmented BNF):

  • RFC 5234 - Augmented BNF for Syntax Specifications: ABNF

  • Extended Backus–Naur form — Wikipedia

  • Augmented Backus–Naur form — Wikipedia

• McKeeman form

  • McKeeman Form

  • grammars-v4/mckeeman-form at master · antlr/grammars-v4

Railroad diagram

• bnf - Tool for generating railroad diagram used on json.org - Stack Overflow

• BNF notation to SVG railroad diagram - Stack Overflow

• tabatkins/railroad-diagrams: A small JS+SVG library for drawing railroad syntax diagrams, like on JSON.org. Now with a Python port!