josh/propane
1
0 Fork 0
Code Issues 3 Pull Requests Projects Releases Wiki Activity

Commit Graph

  • 9dffa3c41a Recursively build item sets josh 2021-08-29 12:38:44 -04:00
  • ceb7e9ee32 Add EOF token to Start rule patterns josh 2021-08-29 11:48:49 -04:00
  • 6026bf1514 Start building following item sets josh 2021-08-29 09:41:00 -04:00
  • 9cc1890ddc One Rule object stores all alternative patterns josh 2021-08-28 10:28:50 -04:00
  • e4f2fffe50 add Item#closed_items josh 2021-08-28 09:47:01 -04:00
  • d931bcb513 Do not expand rules josh 2021-08-28 09:23:08 -04:00
  • 2e16b0bd6e Start on Item and ItemSet josh 2021-08-28 09:02:19 -04:00
  • 6ce94e15af Expand rules josh 2021-08-28 08:11:06 -04:00
  • 3f92ae46c4 Map rule components to Token/Rule references josh 2021-08-22 21:21:41 -04:00
  • 00016f16b3 Combine Grammar and Generator into top-level Imbecile class josh 2021-08-22 21:04:46 -04:00
  • 9273bfccf6 Move Token/Rule out of Grammar class josh 2021-08-19 20:00:40 -04:00
  • f295acb593 Generator builds a Lexer, not a Lexer::DFA josh 2021-08-19 13:11:12 -04:00
  • 51a31317a6 Move FA#build_tables to Lexer::DFA josh 2021-08-19 11:55:34 -04:00
  • 9459883e74 Add Lexer class; Move LexerDFA to Lexer::DFA josh 2021-08-18 17:09:45 -04:00
  • 28591907c1 Move FA class out of Regex class josh 2021-08-18 17:05:03 -04:00
  • 37d6917b49 Rework Rule constructor josh 2021-07-27 21:22:46 -04:00
  • 2685c05360 Change rule syntax josh 2021-07-19 21:55:08 -04:00
  • c0c3353fd7 Test lexing empty null string returns EOF josh 2021-07-06 12:06:07 -04:00
  • 3158e51059 Add length field to LexedToken josh 2021-07-06 11:59:35 -04:00
  • d9e4f64d2e Fix returning TOKEN_EOF when lexing at EOF josh 2021-07-06 11:55:44 -04:00
  • ec2dcf9a72 Fix not progressing through input while lexing a token josh 2021-07-06 11:47:33 -04:00
  • 578e165e2d Fix off-by-one error in state IDs josh 2021-07-06 11:44:03 -04:00
  • e8df4296cc Begin testing lexer josh 2021-07-06 11:09:39 -04:00
  • 230c324209 Fix iterating through all transitions in a state josh 2021-07-06 11:09:13 -04:00
  • 1271e19b50 Test multi-byte code point decoding josh 2021-07-06 11:02:43 -04:00
  • 12e11399af Add decoder tests josh 2021-07-06 10:57:06 -04:00
  • 24fab8515d Decoder.decode_code_point returns struct with code point and length together josh 2021-07-06 10:50:32 -04:00
  • 1dcdd87a28 Generate token constants and names to top-level parser class josh 2021-07-06 10:28:35 -04:00
  • 8aec7ec0de Lexer class can be used standalone josh 2021-07-06 10:15:07 -04:00
  • c96d55b031 Fix class name josh 2021-07-06 10:14:14 -04:00
  • ca7d4862f9 Run test executable; build with unit tests josh 2021-07-06 10:03:42 -04:00
  • 3c874ae4c1 Compile generated parser with a test file josh 2021-07-05 23:05:55 -04:00
  • 748c219625 Do not return dropped tokens from Lexer.lex_token() josh 2021-07-05 22:53:58 -04:00
  • 71ee7de9f9 Remove obsolete lex() and lex_token() methods josh 2021-07-05 22:49:50 -04:00
  • 2121acc87e Complete Lexer.lex_token() josh 2021-07-05 22:41:09 -04:00
  • f2563cf255 Work on Lexer.lex_token() josh 2021-07-05 22:02:27 -04:00
  • 24d12be3b9 Add TOKEN enum entries for EOF, decode error, drop, and none josh 2021-07-05 20:11:55 -04:00
  • 91d6ee25ea Add Lexer class josh 2021-07-05 19:13:41 -04:00
  • 2f1cb47bea Add Decoder class to decode code points josh 2021-07-05 18:47:10 -04:00
  • 651461c570 Start on decode_code_point() josh 2021-06-29 23:17:44 -04:00
  • 3ce54bd303 Start on lex()/lex_token() josh 2021-06-29 23:10:40 -04:00
  • 15454f926a Add TokenNames array josh 2021-06-29 22:49:12 -04:00
  • 4beb3d2016 Add some token constants josh 2021-06-27 23:08:14 -04:00
  • aae7bc188c Use unsigned literals josh 2021-06-26 18:11:20 -04:00
  • a716dedeb6 Start on test framework to compile and run generated parser josh 2021-06-26 16:17:24 -04:00
  • 93cb25df62 Do not generate token names for drop tokens josh 2021-06-26 16:16:18 -04:00
  • 61dd5bc5a0 Move imbecile_spec to lexer_dfa_spec josh 2021-06-26 16:01:49 -04:00
  • 10a8ef5eb4 Update generated lexer state and transition tables josh 2021-06-26 15:58:36 -04:00
  • 98584ce07a Add FA#build_tables josh 2021-06-24 15:06:10 -04:00
  • 2122ca02fe Start generating lexer states and transitions josh 2021-06-23 23:15:02 -04:00
  • 5881f13380 Generate enum of token identifiers josh 2021-06-23 22:22:45 -04:00
  • ebc1d8f001 Fix FA#to_s to show correct destination state josh 2021-06-23 22:21:53 -04:00
  • 5fecd5c6a2 Refactor into FA#enumerate josh 2021-06-22 22:01:39 -04:00
  • 5b688b090d Add some attr_readers josh 2021-06-21 22:52:27 -04:00
  • f77218801f Error if Start rule not found josh 2021-06-21 22:48:17 -04:00
  • 70118dd019 Check for duplicate token/rule names in Generator josh 2021-06-21 22:34:43 -04:00
  • d552f2a540 CLI: accept --log option josh 2021-06-19 12:06:02 -04:00
  • d2fac07249 Add Generator class josh 2021-06-15 16:51:36 -04:00
  • a34272dfd6 Add Grammar::Rule class josh 2021-06-14 22:49:43 -04:00
  • 9d05861819 Parse grammar input by multiline regex josh 2021-06-12 22:57:32 -04:00
  • 03035a25a5 Update spec task to accept an example pattern josh 2021-06-12 22:46:13 -04:00
  • db70f8b94d Add "drop" grammar keyword to drop patterns josh 2021-06-09 22:48:30 -04:00
  • f67dd62b20 Add \s to expand to whitespace characters josh 2021-06-09 22:37:00 -04:00
  • c6bac6d3a1 Rename TokenDFA -> LexerDFA josh 2021-06-08 13:54:46 -04:00
  • aa92970c31 Add some lexer tests josh 2021-06-07 22:21:52 -04:00
  • b8282e748e Start on a test lexer for lexer specs josh 2021-06-07 17:17:37 -04:00
  • 930ac56148 Do not accept 0-length tokens josh 2021-06-06 15:29:30 -04:00
  • 7f54778ba8 Rename Regex::DFA to TokenDFA josh 2021-06-06 15:18:21 -04:00
  • 701903def2 Token should build its own NFA josh 2021-06-06 14:09:28 -04:00
  • afea886ecb Add Grammar::Token class josh 2021-06-06 14:04:33 -04:00
  • 03b2e87186 Grammar takes in input string instead of file name josh 2021-06-06 10:09:53 -04:00
  • e4370cac62 Print accepting token in FA#to_s josh 2021-06-06 09:59:28 -04:00
  • ed3f599e25 Create common FA/State/Transition classes across NFA/DFA josh 2021-06-06 09:41:23 -04:00
  • 1228a76c55 Fix MultiplicityUnit#to_nfa again josh 2021-05-26 10:17:03 -04:00
  • 538e360cb3 Fix MultiplicityUnit#to_nfa josh 2021-05-25 16:59:22 -04:00
  • e7f8c3726c Fix NFA#to_s josh 2021-05-25 16:14:19 -04:00
  • b6e3a5c151 Record accepting token in DFA state josh 2021-05-25 16:00:25 -04:00
  • 35ef94dbd3 Print out DFA to test josh 2021-05-25 15:52:47 -04:00
  • 37e1252ded Continue building DFA josh 2021-05-25 15:44:23 -04:00
  • 214ece7d90 Add NFA::Transition, start on DFA construction josh 2021-05-23 21:41:50 -04:00
  • 8473df421a Add specs for CodePointRange josh 2021-05-23 20:41:40 -04:00
  • 3987f08cd7 Add CodePointRange class josh 2021-05-23 17:52:20 -04:00
  • 3a1650906e Show non-printable characters better in NFA#to_s josh 2021-05-21 14:39:02 -04:00
  • 952bffc33c Move DFA#nil_transition_states to NFA::State josh 2021-05-21 14:27:42 -04:00
  • f64f3683c6 Add NFA#to_s josh 2021-05-21 14:24:16 -04:00
  • 43f5caf449 Fix some NFA creation josh 2021-05-20 17:34:18 -04:00
  • f38a7456e9 Add DFA#nil_transition_states josh 2021-05-20 17:08:34 -04:00
  • c77c81bf25 Mark regex NFA end state as accepting the token josh 2021-05-18 16:34:26 -04:00
  • 7196a0605a Add DFA class josh 2021-05-18 16:31:16 -04:00
  • 24054461a2 Merge Regex::Parser into Regex, move Unit to its own file josh 2021-05-18 16:14:42 -04:00
  • 89a5976064 Make Regex::Parser build a NFA after parsing josh 2021-05-18 16:07:39 -04:00
  • d3df67be1e Update rake josh 2021-05-18 15:55:19 -04:00
  • 791340b292 Build NFA for each token pattern josh 2021-05-17 22:57:18 -04:00
  • cf8718b69c Allow token definition with no pattern josh 2021-05-17 22:40:23 -04:00
  • 39f164a7db Parse . in a regex josh 2021-05-17 17:20:56 -04:00
  • 70b3e56de2 Store all characters as ranges; add CharacterClassUnit#to_nfa josh 2021-05-14 13:52:03 -04:00
  • 2e8e72a1e8 Add CharacterClassUnit and use it instead of AlternatesUnit josh 2021-05-14 12:32:53 -04:00
  • ea27baa630 Add #to_nfa for other regex unit types josh 2021-05-13 15:57:09 -04:00
  • d8dd64d860 Add NFA class josh 2021-05-13 00:01:12 -04:00
  • 54cefda186 Use Parser josh 2021-05-11 16:52:28 -04:00