Incremental Parsing with Combinators Jean-Philippe Bernardy Talk given at Chalmers FP workshop, 2008.

Yet Another Parsing Library? ● Incremental – Real-time feedback in editor – Performance should not depend on size of file ● Combinators – Users build their own grammars – Grammars as libraries ● Demo

The Plan ● Online parsers exist – No need to parse everything to look at the beginning of the tree – Solves half of the problem ● Improve to solve the 2 nd half – Save intermediate parsing states, resume.

$$ :+:+ 57 Parser “5+7 ” Run :+ 57 Online (Polish) Parsers

Questions?

Save and resume: idea ● pre-compute intermediate results at every “page” – lazily ● incremental update: – reuse the pre-computed results that don't depend on the input – recompute the invalidated ones – tree provided to the user is computed “online”, from the most relevant intermediate result

Save and resume: diagram ( *( 6 +7 * 6) ( *( 6 +7 * 6) Apply online algorithm using this intermediate result.

Save and resume: problem ● Parser provides no access to the intermediate states ● Run function is processes the steps in “right associative” way (foldr-like) ● This is necessary for online behaviour ● Code

Save and resume: solution ● Suspensions ● Left-eval (foldl like) ● Code $$ :+:+ 57

Questions?

Problem: linear structures ● Most programs have a high-level linear structure ● A precomputed list prefix is not so useful: finishing it is still costly: O(prefix length) ● Fortunately, most programs don't have too many top- level nodes.

Problem: visiting the tree ● Given a full parse tree, we want to find node at a given index. – Ok with a linear search – Binary search will force the whole tree! : :1 : :3 2 []4

Conclusions ● No update of parse tree! – and still incremental ● No startup cost ● Cool usage of lazy evaluation ● User code must use the parse tree lazily – No safeguard against carelessness