josh/propane
1
0 Fork 0
Code Issues 3 Pull Requests Projects Releases Wiki Activity

Commit Graph

  • 058945e08b Add Token struct with validity checks josh 2023-03-13 16:14:50 -04:00
  • f402315201 Name internally generated tokens with double-underscore josh 2023-03-13 14:13:13 -04:00
  • d10e982fc8 Reserve double-underscore identifiers for internal use josh 2023-03-13 14:12:12 -04:00
  • 64974cc1e2 Refactor some of Lexer.attempt_lex_token() into find_longest_match() josh 2023-03-12 21:19:03 -04:00
  • b92679e0c2 Replace LexedToken with Result struct josh 2023-03-11 21:16:55 -05:00
  • 1af018b103 MatchInfo: store pointer to the accepting state rather than copying its fields josh 2023-03-09 20:16:57 -05:00
  • aabb574fea Use CodePoint type for code point values josh 2023-03-09 19:10:17 -05:00
  • 30004c571d Use result type query methods on Decoder.Result josh 2023-03-09 19:04:15 -05:00
  • b00f53050b Clean up Decoder result handling josh 2023-03-06 20:32:01 -05:00
  • 2e5083d823 Add explicit base types to the enums josh 2023-03-06 19:58:24 -05:00
  • 7f844711d9 Start on more of the user guide josh 2023-02-28 20:53:31 -05:00
  • e2100f6ccd Move JSON parser source to standalone file josh 2023-01-16 14:59:43 -05:00
  • 3057173370 Update license copyright year josh 2023-01-04 20:42:15 -05:00
  • 90d08e3554 Add script to build distributable propane script josh 2023-01-04 20:41:58 -05:00
  • f5756e9523 Add script to generate user guide HTML josh 2023-01-03 22:17:16 -05:00
  • dbc5560aec Add JSON parser test case josh 2022-10-16 20:17:48 -04:00
  • 62451f3a92 Allow multiple test files to be compiled josh 2022-11-14 21:00:09 -05:00
  • 2e48921bb1 Fix $n expansion for rules to use rule component ptype josh 2022-11-14 20:58:47 -05:00
  • c6ea4f83c2 Store parser values according to the rule/pattern type josh 2022-11-13 22:20:30 -05:00
  • e4a160f918 Allow defining multiple parser types and assigning parser types to tokens and rules josh 2022-11-13 13:35:19 -05:00
  • 8dc27686aa Rename result_type to ptype, parser result to pvalue josh 2022-11-12 13:29:10 -05:00
  • bca0a14371 Allow storing a result value for a token from a lexer code block josh 2022-10-16 21:40:25 -04:00
  • ca8a360c0e Provide matched text to lexer user code block josh 2022-10-15 13:37:34 -04:00
  • 623c644e74 Work on real D strings instead of ubyte pointer and length josh 2022-10-15 13:32:01 -04:00
  • de93d23585 Add test for non-LALR grammar failing to generate parser josh 2022-10-13 05:17:06 -04:00
  • ad09ff039a Add spec to test parsing lists josh 2022-10-13 05:02:05 -04:00
  • 727c8cd1ea Execute rule user code blocks when reducing the rule josh 2022-10-13 04:56:50 -04:00
  • 74beaf7ed8 Store final parse result and give access with .result parser property josh 2022-10-12 21:33:09 -04:00
  • 31970522de Store parse result; add result_type grammar keyword josh 2022-10-12 20:56:14 -04:00
  • 02be6de48e Add lexer modes and $mode() code expansion josh 2022-10-09 22:49:01 -04:00
  • b2d11321fe Add grammar syntax to specify lexer mode for tokens and patterns josh 2022-10-04 22:23:39 -04:00
  • 6bd9d4a09b Remove _TOKEN_NONE and use _TOKEN_COUNT instead josh 2022-10-03 21:40:34 -04:00
  • ced5a56424 Support global user code blocks josh 2022-10-03 21:24:13 -04:00
  • 66d654b6b9 Add $token() user code block expansion josh 2022-10-02 10:43:47 -04:00
  • 43fb74fe4b Capture and verify stdout results from tests josh 2022-10-02 10:31:07 -04:00
  • 01ef4fc27c Rename some test files josh 2022-10-02 10:10:04 -04:00
  • f46b5b3f4d Remove TOKEN_EOF; define EOF token and start rule in Generator josh 2022-10-02 10:07:44 -04:00
  • 150be33826 Assign pattern code IDs in Generator instead of Grammar josh 2022-10-01 10:36:35 -04:00
  • e7e30c4f28 Add pattern statement josh 2022-09-30 21:05:18 -04:00
  • 04367db0ac Add forward slashes around patterns and parse more robustly josh 2022-09-28 23:05:01 -04:00
  • 1547528ecf Split grammar statement parsing into separate methods josh 2022-09-27 18:49:04 -04:00
  • 62cb5c03f2 Refactor grammar parsing a bit josh 2022-09-27 15:28:39 -04:00
  • 4b148f3797 Add Grammar spec josh 2022-09-27 14:45:29 -04:00
  • a4c5546876 Disable parser debug output josh 2022-09-27 12:33:23 -04:00
  • 48704295bb Rename lexer state "accepts" to "token" josh 2022-09-25 14:57:46 -04:00
  • 4d716f6c10 Remove Token#c_name; use given token case in token constants josh 2022-09-25 14:48:49 -04:00
  • 01c9340819 Rename lexer variables to reflect trying to get longest match josh 2022-09-25 14:44:44 -04:00
  • 672098ad32 Execute user code blocks assigned to tokens josh 2022-09-24 17:31:40 -04:00
  • 92ce30f354 Add tokenid keyword josh 2022-09-19 22:16:57 -04:00
  • 38ae5ac7a1 Split Token class into Token/Pattern josh 2022-09-15 22:46:44 -04:00
  • bf075a69f6 Test matching a semicolon josh 2022-07-23 22:25:17 -04:00
  • b682c72b17 Add semicolon to end of all grammar statements josh 2022-07-23 22:09:19 -04:00
  • 382e17804c Test SLR grammar josh 2022-06-27 21:06:03 -04:00
  • 30f4cfcc99 Write parser log file josh 2022-06-26 11:06:55 -04:00
  • 2fbe13e071 Do not consume lookahead token when reducing josh 2022-06-25 21:35:54 -04:00
  • f2cc5b112e Handle shifting states after reducing josh 2022-06-25 16:16:20 -04:00
  • 84c4a16ce6 Start on Parser.parse() josh 2022-06-21 23:03:00 -04:00
  • f17efe8c82 Add RuleSet#id to use when reducing josh 2022-06-21 20:07:27 -04:00
  • 60e2818075 Determine the reduce actions for each parser state josh 2022-06-17 01:45:48 -04:00
  • 5af3179ff2 Add RuleSet#start_token_set josh 2022-06-15 23:27:19 -04:00
  • 11ea02fb86 Determine the possibly-empty RuleSets josh 2022-06-14 23:32:07 -04:00
  • 8f38d0b017 Clean up ItemSet#close! josh 2022-06-14 23:01:05 -04:00
  • a91bf2f250 Clean up parser ItemSet processing josh 2022-06-14 22:57:07 -04:00
  • 0f1c00d1a6 Add RuleSet#could_be_empty? josh 2022-06-14 07:35:58 -04:00
  • 291509f005 Add Rule#empty? josh 2022-06-14 07:35:38 -04:00
  • c1769503a8 Add Item#complete? josh 2022-06-14 07:35:07 -04:00
  • 8b152abaa7 Add ItemSet#leading_item_sets josh 2022-06-14 07:34:35 -04:00
  • d6e5c4325d Add new start rule ending with EOF token rather than appending EOF token to all user-specified start rules josh 2022-06-07 22:25:31 -04:00
  • 2837dfda6b Add Rule#id josh 2022-06-06 22:30:28 -04:00
  • 9d850294a9 Clean up and comment Item and ItemSet josh 2022-06-05 18:12:14 -04:00
  • df8088c3c6 Clean up rule format in grammar files josh 2022-06-05 16:28:35 -04:00
  • ba74d0a20a Reduce maximum code point value to not interfere with magic code point values used by parser josh 2022-06-05 15:24:40 -04:00
  • fe607291f4 Use .propane extension for test grammars josh 2022-06-05 15:18:55 -04:00
  • c42c3576a5 Remove RuleSet creation from Grammar josh 2022-06-05 15:16:41 -04:00
  • ca6a93a4c5 Move generation logic to new Generator class josh 2022-06-05 14:57:58 -04:00
  • b1d5cddab4 Change Token constructor to take a Hash of all parameters josh 2022-06-05 14:40:41 -04:00
  • f37801ec9e Store tokens and drop tokens separately josh 2022-06-05 14:36:19 -04:00
  • 6f1ce32775 Move Lexer::DFA#build_tables to Lexer josh 2022-06-05 14:18:35 -04:00
  • 57a3e9d9f6 Move grammar parsing into new Grammar class josh 2022-06-05 10:23:34 -04:00
  • 34eb1370ff Move RuleSet::Patern to top-level Rule class josh 2022-06-04 17:49:36 -04:00
  • 7598c589fe Detect other invalid UTF-8 encodings josh 2022-05-31 22:26:09 -04:00
  • a0af8b0d7c Rename Rule to RuleSet josh 2022-05-30 15:49:12 -04:00
  • 164a4854fb Update README josh 2022-05-30 15:40:31 -04:00
  • ddadc2008b Rename to propane josh 2022-05-28 20:20:03 -04:00
  • fbd215098b Update license years josh 2022-05-27 21:49:54 -04:00
  • bfe2916165 Update bundler josh 2022-05-27 00:15:03 -04:00
  • c9bc4832f4 bundle update josh 2022-05-27 00:14:26 -04:00
  • 6dfef8573f Fix ERB constructor call for Ruby 3.2 warnings josh 2022-05-27 00:12:40 -04:00
  • f3ed678fe1 Store tokens in Hash by name josh 2021-09-27 21:40:12 -04:00
  • 280b749e38 Track Rule IDs josh 2021-09-27 21:29:44 -04:00
  • d6779aef00 Start on Parser#build_tables josh 2021-09-22 23:26:36 -04:00
  • 746ec89be8 Add test for a rule that can be arrived at from multiple states josh 2021-09-21 21:40:11 -04:00
  • 997f34a1e4 Keep track of item set in-links josh 2021-09-21 21:25:45 -04:00
  • a2795bb531 Keep track of follow item sets by symbol for each item set josh 2021-09-21 17:09:53 -04:00
  • 850e639e3a update identical rule spec to use lookahead symbol josh 2021-09-06 20:18:17 -04:00
  • 5f7e548fe3 Remove Rule::Pattern, Item stores a Rule reference josh 2021-09-06 19:41:29 -04:00
  • bdb10e7afc test duplicate rules josh 2021-09-05 09:50:04 -04:00
  • 7bdaf7cdbc Do not create item set following EOF token josh 2021-09-05 07:51:59 -04:00
  • 08e3516ad9 Add wikipedia LR(0) parser example test josh 2021-09-04 22:33:29 -04:00
  • 2c8f3c6e9a Avoid infinite loop with self-referential rules josh 2021-09-04 22:29:10 -04:00