[{"content":"Intro This post is inspired by the amazing Guido van Rossum series on PEG building and follows my idea of building simple version of B-Minor language.\nI have already some experience in building simple recusrive descent parsers, so this is a good time to try to do something more complicated and PEG seems like a good choice. And as I am still in learning mode, I want to implement it in Zig, since I like a lot manual allocations and many other features of this language (but of course there are many things that makes me quite angry).\nBut enought talks, lets dive in.\nPlan Here is the plan and constraints on how I am going to proceed.\nImplement basic arithmetic PEG parser, since rules for it are already provided in this nice Wikipedia page Add memoizing functionality. Using this basic PEG parser as a foundation, implement compiler-compiler, which uses PEG rules and generates the same parser. Add more rules and as a result implement full B-Minor parser. Basic Arithmetic parser We have following rules\nExpr ← Sum Sum ← Product ((\u0026#39;+\u0026#39; / \u0026#39;-\u0026#39;) Product)* Product ← Power ((\u0026#39;*\u0026#39; / \u0026#39;/\u0026#39;) Power)* Power ← Value (\u0026#39;^\u0026#39; Power)? Value ← NUMBER / \u0026#39;(\u0026#39; Expr \u0026#39;)\u0026#39; Here NUMBER is a token, which corresponds to the [0-9]+ expression. Of course, it is possible to make tokenization part of the grammar, but I am not completely convinced that it is that useful, so I am ok with extra tokens here and there.\nAlso, while it is useful to have tokens like \u0026lsquo;+\u0026rsquo;/\u0026rsquo;-\u0026rsquo; etc in rules, it is more convinient to have them defined as proper tags in tokenizer.\nTokenizer Tokenizer is straightforward and very similar to what is written in Python tokenizer version. Main difference is that since I do not have pre-ready tokenizer, I split tokenizer in two parts: Scanner which does all the heavy-lifintg, but rather dumb and Lexer which is basically Scanner with buffer and allow me later to utilize infinite backtracking.\npub const Lexer = struct { scanner: Scanner, tokens: std.ArrayList(Token), pos: u32, const Self = @This(); pub fn init(allocator: std.mem.Allocator, scanner: Scanner) Self { const tokens = std.ArrayList(Token).init(allocator); return .{ .scanner = scanner, .tokens = tokens, .pos = 0 }; } pub fn mark(self: Self) u32 { return self.pos; } pub fn reset(self: *Self, pos: u32) void { self.pos = pos; } pub fn nextToken(self: *Self) !Token { const token = try self.peekToken(); self.pos += 1; return token; } pub fn peekToken(self: *Self) !Token { if (self.pos \u0026gt;= self.tokens.items.len) { try self.tokens.append(self.scanner.nextToken()); } return self.tokens.items[self.pos]; } }; Of course, at some point I should add errors resolution. It is nice to ride on auto inference, but as we know, Zig is not good at resolving error unions from recursive calls and we have plenty of those. Alternative is to panic all the time, since most of the important errors arises from memory manipulation in ArrayList and there is not much I can do about it anyway.\nAnother thing is that you do not find any deinit in this code, since it is assumed, that Arena Allocator is used. Of course, it is possible to sprinkle everything with destroy, free and deinit just in case, but I do not see it as a reasonable thing. It only adds extra burden of tracking memory leaks, but parsing is a one-time procedure, so all memory should be released anyway. It is faster, it is easier, it is more reliable and it is one the things I love so nuch about Zig.\nMemoizing Needs to figure out, how to do it in Zig.\n","permalink":"https://arkoniak.com/posts/zig-peg-parser/","summary":"Building a PEG parser in Zig, inspired by Guido van Rossum\u0026rsquo;s PEG series — first step towards a full B-Minor compiler.","title":"Zig Peg Parser"},{"content":"Introduction This is bold text, and this is emphasized text.\nThis is a test to verify that this post can be easily updated.\n","permalink":"https://arkoniak.com/posts/my-first-post/","summary":"A test post to check that publishing works.","title":"My First Post"},{"content":"Hi, I\u0026rsquo;m \u0026hellip; — \u0026hellip;\nWhat this blog is about Programming languages and parsers (currently: PEG parsers in Zig) \u0026hellip; Elsewhere \u0026hellip; ","permalink":"https://arkoniak.com/about/","summary":"\u003cp\u003eHi, I\u0026rsquo;m \u0026hellip; — \u0026hellip;\u003c/p\u003e\n\u003ch2 id=\"what-this-blog-is-about\"\u003eWhat this blog is about\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eProgramming languages and parsers (currently: PEG parsers in Zig)\u003c/li\u003e\n\u003cli\u003e\u0026hellip;\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"elsewhere\"\u003eElsewhere\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003e\u0026hellip;\u003c/li\u003e\n\u003c/ul\u003e","title":"About"}]