 josh
|
fbd215098b
|
Update license years
|
2022-05-27 21:49:54 -04:00 |
|
|
josh
|
bfe2916165
|
Update bundler
|
2022-05-27 00:15:03 -04:00 |
|
|
josh
|
c9bc4832f4
|
bundle update
|
2022-05-27 00:14:26 -04:00 |
|
|
josh
|
6dfef8573f
|
Fix ERB constructor call for Ruby 3.2 warnings
|
2022-05-27 00:12:40 -04:00 |
|
|
josh
|
f3ed678fe1
|
Store tokens in Hash by name
|
2021-09-27 21:40:12 -04:00 |
|
|
josh
|
280b749e38
|
Track Rule IDs
|
2021-09-27 21:29:44 -04:00 |
|
|
josh
|
d6779aef00
|
Start on Parser#build_tables
|
2021-09-22 23:26:36 -04:00 |
|
|
josh
|
746ec89be8
|
Add test for a rule that can be arrived at from multiple states
|
2021-09-21 21:40:11 -04:00 |
|
|
josh
|
997f34a1e4
|
Keep track of item set in-links
|
2021-09-21 21:32:18 -04:00 |
|
|
josh
|
a2795bb531
|
Keep track of follow item sets by symbol for each item set
|
2021-09-21 17:09:53 -04:00 |
|
|
josh
|
850e639e3a
|
update identical rule spec to use lookahead symbol
|
2021-09-06 20:18:17 -04:00 |
|
|
josh
|
5f7e548fe3
|
Remove Rule::Pattern, Item stores a Rule reference
|
2021-09-06 19:41:29 -04:00 |
|
|
josh
|
bdb10e7afc
|
test duplicate rules
|
2021-09-05 09:50:04 -04:00 |
|
|
josh
|
7bdaf7cdbc
|
Do not create item set following EOF token
|
2021-09-05 07:51:59 -04:00 |
|
|
josh
|
08e3516ad9
|
Add wikipedia LR(0) parser example test
|
2021-09-04 22:33:34 -04:00 |
|
|
josh
|
2c8f3c6e9a
|
Avoid infinite loop with self-referential rules
|
2021-09-04 22:29:10 -04:00 |
|
|
josh
|
9dffa3c41a
|
Recursively build item sets
|
2021-08-29 12:38:44 -04:00 |
|
|
josh
|
ceb7e9ee32
|
Add EOF token to Start rule patterns
|
2021-08-29 11:48:49 -04:00 |
|
|
josh
|
6026bf1514
|
Start building following item sets
|
2021-08-29 09:41:00 -04:00 |
|
|
josh
|
9cc1890ddc
|
One Rule object stores all alternative patterns
|
2021-08-28 10:28:50 -04:00 |
|
|
josh
|
e4f2fffe50
|
add Item#closed_items
|
2021-08-28 09:47:01 -04:00 |
|
|
josh
|
d931bcb513
|
Do not expand rules
|
2021-08-28 09:23:08 -04:00 |
|
|
josh
|
2e16b0bd6e
|
Start on Item and ItemSet
|
2021-08-28 09:02:19 -04:00 |
|
|
josh
|
6ce94e15af
|
Expand rules
|
2021-08-28 08:11:06 -04:00 |
|
|
josh
|
3f92ae46c4
|
Map rule components to Token/Rule references
|
2021-08-22 21:21:41 -04:00 |
|
|
josh
|
00016f16b3
|
Combine Grammar and Generator into top-level Imbecile class
|
2021-08-22 21:04:46 -04:00 |
|
|
josh
|
9273bfccf6
|
Move Token/Rule out of Grammar class
|
2021-08-19 20:00:40 -04:00 |
|
|
josh
|
f295acb593
|
Generator builds a Lexer, not a Lexer::DFA
|
2021-08-19 13:11:12 -04:00 |
|
|
josh
|
51a31317a6
|
Move FA#build_tables to Lexer::DFA
|
2021-08-19 11:55:34 -04:00 |
|
|
josh
|
9459883e74
|
Add Lexer class; Move LexerDFA to Lexer::DFA
|
2021-08-18 17:09:45 -04:00 |
|
|
josh
|
28591907c1
|
Move FA class out of Regex class
|
2021-08-18 17:05:03 -04:00 |
|
|
josh
|
37d6917b49
|
Rework Rule constructor
|
2021-07-27 21:22:46 -04:00 |
|
|
josh
|
2685c05360
|
Change rule syntax
|
2021-07-19 21:55:08 -04:00 |
|
|
josh
|
c0c3353fd7
|
Test lexing empty null string returns EOF
|
2021-07-06 12:06:07 -04:00 |
|
|
josh
|
3158e51059
|
Add length field to LexedToken
|
2021-07-06 11:59:35 -04:00 |
|
|
josh
|
d9e4f64d2e
|
Fix returning TOKEN_EOF when lexing at EOF
|
2021-07-06 11:55:44 -04:00 |
|
|
josh
|
ec2dcf9a72
|
Fix not progressing through input while lexing a token
|
2021-07-06 11:47:33 -04:00 |
|
|
josh
|
578e165e2d
|
Fix off-by-one error in state IDs
|
2021-07-06 11:44:03 -04:00 |
|
|
josh
|
e8df4296cc
|
Begin testing lexer
|
2021-07-06 11:09:39 -04:00 |
|
|
josh
|
230c324209
|
Fix iterating through all transitions in a state
|
2021-07-06 11:09:13 -04:00 |
|
|
josh
|
1271e19b50
|
Test multi-byte code point decoding
|
2021-07-06 11:02:43 -04:00 |
|
|
josh
|
12e11399af
|
Add decoder tests
|
2021-07-06 10:57:06 -04:00 |
|
|
josh
|
24fab8515d
|
Decoder.decode_code_point returns struct with code point and length together
|
2021-07-06 10:50:32 -04:00 |
|
|
josh
|
1dcdd87a28
|
Generate token constants and names to top-level parser class
|
2021-07-06 10:28:35 -04:00 |
|
|
josh
|
8aec7ec0de
|
Lexer class can be used standalone
|
2021-07-06 10:15:07 -04:00 |
|
|
josh
|
c96d55b031
|
Fix class name
|
2021-07-06 10:14:14 -04:00 |
|
|
josh
|
ca7d4862f9
|
Run test executable; build with unit tests
|
2021-07-06 10:03:42 -04:00 |
|
|
josh
|
3c874ae4c1
|
Compile generated parser with a test file
|
2021-07-05 23:05:55 -04:00 |
|
|
josh
|
748c219625
|
Do not return dropped tokens from Lexer.lex_token()
|
2021-07-05 22:53:58 -04:00 |
|
|
josh
|
71ee7de9f9
|
Remove obsolete lex() and lex_token() methods
|
2021-07-05 22:49:50 -04:00 |
|
|
josh
|
2121acc87e
|
Complete Lexer.lex_token()
|
2021-07-05 22:41:09 -04:00 |
|
|
josh
|
f2563cf255
|
Work on Lexer.lex_token()
|
2021-07-05 22:02:27 -04:00 |
|
|
josh
|
24d12be3b9
|
Add TOKEN enum entries for EOF, decode error, drop, and none
|
2021-07-05 20:11:55 -04:00 |
|
|
josh
|
91d6ee25ea
|
Add Lexer class
|
2021-07-05 19:13:41 -04:00 |
|
|
josh
|
2f1cb47bea
|
Add Decoder class to decode code points
|
2021-07-05 18:47:10 -04:00 |
|
|
josh
|
651461c570
|
Start on decode_code_point()
|
2021-06-29 23:17:44 -04:00 |
|
|
josh
|
3ce54bd303
|
Start on lex()/lex_token()
|
2021-06-29 23:10:40 -04:00 |
|
|
josh
|
15454f926a
|
Add TokenNames array
|
2021-06-29 22:54:24 -04:00 |
|
|
josh
|
4beb3d2016
|
Add some token constants
|
2021-06-27 23:09:42 -04:00 |
|
|
josh
|
aae7bc188c
|
Use unsigned literals
|
2021-06-26 18:11:20 -04:00 |
|
|
josh
|
a716dedeb6
|
Start on test framework to compile and run generated parser
|
2021-06-26 16:17:24 -04:00 |
|
|
josh
|
93cb25df62
|
Do not generate token names for drop tokens
|
2021-06-26 16:16:18 -04:00 |
|
|
josh
|
61dd5bc5a0
|
Move imbecile_spec to lexer_dfa_spec
|
2021-06-26 16:01:49 -04:00 |
|
|
josh
|
10a8ef5eb4
|
Update generated lexer state and transition tables
|
2021-06-26 15:58:36 -04:00 |
|
|
josh
|
98584ce07a
|
Add FA#build_tables
|
2021-06-24 15:06:10 -04:00 |
|
|
josh
|
2122ca02fe
|
Start generating lexer states and transitions
|
2021-06-23 23:15:02 -04:00 |
|
|
josh
|
5881f13380
|
Generate enum of token identifiers
|
2021-06-23 22:22:45 -04:00 |
|
|
josh
|
ebc1d8f001
|
Fix FA#to_s to show correct destination state
|
2021-06-23 22:21:53 -04:00 |
|
|
josh
|
5fecd5c6a2
|
Refactor into FA#enumerate
|
2021-06-22 22:01:39 -04:00 |
|
|
josh
|
5b688b090d
|
Add some attr_readers
|
2021-06-21 22:52:27 -04:00 |
|
|
josh
|
f77218801f
|
Error if Start rule not found
|
2021-06-21 22:48:17 -04:00 |
|
|
josh
|
70118dd019
|
Check for duplicate token/rule names in Generator
|
2021-06-21 22:34:43 -04:00 |
|
|
josh
|
d552f2a540
|
CLI: accept --log option
|
2021-06-19 12:06:02 -04:00 |
|
|
josh
|
d2fac07249
|
Add Generator class
|
2021-06-15 16:51:36 -04:00 |
|
|
josh
|
a34272dfd6
|
Add Grammar::Rule class
|
2021-06-14 22:49:43 -04:00 |
|
|
josh
|
9d05861819
|
Parse grammar input by multiline regex
|
2021-06-12 22:57:32 -04:00 |
|
|
josh
|
03035a25a5
|
Update spec task to accept an example pattern
|
2021-06-12 22:46:13 -04:00 |
|
|
josh
|
db70f8b94d
|
Add "drop" grammar keyword to drop patterns
|
2021-06-09 22:48:30 -04:00 |
|
|
josh
|
f67dd62b20
|
Add \s to expand to whitespace characters
|
2021-06-09 22:37:00 -04:00 |
|
|
josh
|
c6bac6d3a1
|
Rename TokenDFA -> LexerDFA
|
2021-06-08 13:54:46 -04:00 |
|
|
josh
|
aa92970c31
|
Add some lexer tests
|
2021-06-07 22:21:52 -04:00 |
|
|
josh
|
b8282e748e
|
Start on a test lexer for lexer specs
|
2021-06-07 17:17:37 -04:00 |
|
|
josh
|
930ac56148
|
Do not accept 0-length tokens
|
2021-06-06 15:29:30 -04:00 |
|
|
josh
|
7f54778ba8
|
Rename Regex::DFA to TokenDFA
|
2021-06-06 15:18:21 -04:00 |
|
|
josh
|
701903def2
|
Token should build its own NFA
|
2021-06-06 14:09:28 -04:00 |
|
|
josh
|
afea886ecb
|
Add Grammar::Token class
|
2021-06-06 14:04:33 -04:00 |
|
|
josh
|
03b2e87186
|
Grammar takes in input string instead of file name
|
2021-06-06 10:09:53 -04:00 |
|
|
josh
|
e4370cac62
|
Print accepting token in FA#to_s
|
2021-06-06 09:59:28 -04:00 |
|
|
josh
|
ed3f599e25
|
Create common FA/State/Transition classes across NFA/DFA
|
2021-06-06 09:41:23 -04:00 |
|
|
josh
|
1228a76c55
|
Fix MultiplicityUnit#to_nfa again
|
2021-05-26 10:17:03 -04:00 |
|
|
josh
|
538e360cb3
|
Fix MultiplicityUnit#to_nfa
|
2021-05-25 16:59:22 -04:00 |
|
|
josh
|
e7f8c3726c
|
Fix NFA#to_s
|
2021-05-25 16:14:19 -04:00 |
|
|
josh
|
b6e3a5c151
|
Record accepting token in DFA state
|
2021-05-25 16:00:25 -04:00 |
|
|
josh
|
35ef94dbd3
|
Print out DFA to test
|
2021-05-25 15:52:47 -04:00 |
|
|
josh
|
37e1252ded
|
Continue building DFA
|
2021-05-25 15:44:23 -04:00 |
|
|
josh
|
214ece7d90
|
Add NFA::Transition, start on DFA construction
|
2021-05-23 21:41:50 -04:00 |
|
|
josh
|
8473df421a
|
Add specs for CodePointRange
|
2021-05-23 20:41:40 -04:00 |
|
|
josh
|
3987f08cd7
|
Add CodePointRange class
|
2021-05-23 17:52:20 -04:00 |
|
|
josh
|
3a1650906e
|
Show non-printable characters better in NFA#to_s
|
2021-05-21 14:39:02 -04:00 |
|
|
josh
|
952bffc33c
|
Move DFA#nil_transition_states to NFA::State
|
2021-05-21 14:27:42 -04:00 |
|