 josh
|
66d654b6b9
|
Add $token() user code block expansion
|
2022-10-02 10:43:47 -04:00 |
|
|
josh
|
43fb74fe4b
|
Capture and verify stdout results from tests
|
2022-10-02 10:31:07 -04:00 |
|
|
josh
|
01ef4fc27c
|
Rename some test files
|
2022-10-02 10:10:04 -04:00 |
|
|
josh
|
f46b5b3f4d
|
Remove TOKEN_EOF; define EOF token and start rule in Generator
|
2022-10-02 10:07:44 -04:00 |
|
|
josh
|
150be33826
|
Assign pattern code IDs in Generator instead of Grammar
|
2022-10-01 10:36:35 -04:00 |
|
|
josh
|
e7e30c4f28
|
Add pattern statement
|
2022-09-30 21:05:18 -04:00 |
|
|
josh
|
04367db0ac
|
Add forward slashes around patterns and parse more robustly
|
2022-09-28 23:05:01 -04:00 |
|
|
josh
|
1547528ecf
|
Split grammar statement parsing into separate methods
|
2022-09-27 18:49:04 -04:00 |
|
|
josh
|
62cb5c03f2
|
Refactor grammar parsing a bit
|
2022-09-27 15:28:39 -04:00 |
|
|
josh
|
4b148f3797
|
Add Grammar spec
|
2022-09-27 14:45:29 -04:00 |
|
|
josh
|
a4c5546876
|
Disable parser debug output
|
2022-09-27 12:33:23 -04:00 |
|
|
josh
|
48704295bb
|
Rename lexer state "accepts" to "token"
|
2022-09-25 14:57:46 -04:00 |
|
|
josh
|
4d716f6c10
|
Remove Token#c_name; use given token case in token constants
|
2022-09-25 14:48:49 -04:00 |
|
|
josh
|
01c9340819
|
Rename lexer variables to reflect trying to get longest match
|
2022-09-25 14:44:44 -04:00 |
|
|
josh
|
672098ad32
|
Execute user code blocks assigned to tokens
|
2022-09-24 17:31:40 -04:00 |
|
|
josh
|
92ce30f354
|
Add tokenid keyword
|
2022-09-19 22:16:57 -04:00 |
|
|
josh
|
38ae5ac7a1
|
Split Token class into Token/Pattern
|
2022-09-15 22:46:44 -04:00 |
|
|
josh
|
bf075a69f6
|
Test matching a semicolon
|
2022-07-23 22:25:17 -04:00 |
|
|
josh
|
b682c72b17
|
Add semicolon to end of all grammar statements
|
2022-07-23 22:09:19 -04:00 |
|
|
josh
|
382e17804c
|
Test SLR grammar
|
2022-06-27 21:06:03 -04:00 |
|
|
josh
|
30f4cfcc99
|
Write parser log file
Fix bug of skipping rule set IDs.
Remove unneeded out_sets from ItemSet class.
|
2022-06-26 11:06:55 -04:00 |
|
|
josh
|
2fbe13e071
|
Do not consume lookahead token when reducing
|
2022-06-25 21:35:54 -04:00 |
|
|
josh
|
f2cc5b112e
|
Handle shifting states after reducing
|
2022-06-25 16:16:20 -04:00 |
|
|
josh
|
84c4a16ce6
|
Start on Parser.parse()
|
2022-06-21 23:03:00 -04:00 |
|
|
josh
|
f17efe8c82
|
Add RuleSet#id to use when reducing
Parser will know what state to go to after reducing a Rule based on the
RuleSet ID.
Start on Parser class.
|
2022-06-21 20:07:27 -04:00 |
|
|
josh
|
60e2818075
|
Determine the reduce actions for each parser state
|
2022-06-17 01:45:48 -04:00 |
|
|
josh
|
5af3179ff2
|
Add RuleSet#start_token_set
|
2022-06-17 01:35:02 -04:00 |
|
|
josh
|
11ea02fb86
|
Determine the possibly-empty RuleSets
|
2022-06-17 01:35:02 -04:00 |
|
|
josh
|
8f38d0b017
|
Clean up ItemSet#close!
|
2022-06-17 01:35:02 -04:00 |
|
|
josh
|
a91bf2f250
|
Clean up parser ItemSet processing
|
2022-06-14 22:57:07 -04:00 |
|
|
josh
|
0f1c00d1a6
|
Add RuleSet#could_be_empty?
|
2022-06-14 07:35:58 -04:00 |
|
|
josh
|
291509f005
|
Add Rule#empty?
|
2022-06-14 07:35:38 -04:00 |
|
|
josh
|
c1769503a8
|
Add Item#complete?
|
2022-06-14 07:35:07 -04:00 |
|
|
josh
|
8b152abaa7
|
Add ItemSet#leading_item_sets
|
2022-06-14 07:34:35 -04:00 |
|
|
josh
|
d6e5c4325d
|
Add new start rule ending with EOF token rather than appending EOF token to all user-specified start rules
|
2022-06-07 22:25:31 -04:00 |
|
|
josh
|
2837dfda6b
|
Add Rule#id
|
2022-06-06 22:30:53 -04:00 |
|
|
josh
|
9d850294a9
|
Clean up and comment Item and ItemSet
|
2022-06-05 18:12:14 -04:00 |
|
|
josh
|
df8088c3c6
|
Clean up rule format in grammar files
|
2022-06-05 16:28:35 -04:00 |
|
|
josh
|
ba74d0a20a
|
Reduce maximum code point value to not interfere with magic code point values used by parser
|
2022-06-05 15:24:40 -04:00 |
|
|
josh
|
fe607291f4
|
Use .propane extension for test grammars
|
2022-06-05 15:18:55 -04:00 |
|
|
josh
|
c42c3576a5
|
Remove RuleSet creation from Grammar
|
2022-06-05 15:16:41 -04:00 |
|
|
josh
|
ca6a93a4c5
|
Move generation logic to new Generator class
|
2022-06-05 14:57:58 -04:00 |
|
|
josh
|
b1d5cddab4
|
Change Token constructor to take a Hash of all parameters
|
2022-06-05 14:40:41 -04:00 |
|
|
josh
|
f37801ec9e
|
Store tokens and drop tokens separately
|
2022-06-05 14:36:19 -04:00 |
|
|
josh
|
6f1ce32775
|
Move Lexer::DFA#build_tables to Lexer
|
2022-06-05 14:18:35 -04:00 |
|
|
josh
|
57a3e9d9f6
|
Move grammar parsing into new Grammar class
|
2022-06-05 10:23:34 -04:00 |
|
|
josh
|
34eb1370ff
|
Move RuleSet::Patern to top-level Rule class
|
2022-06-04 17:50:26 -04:00 |
|
|
josh
|
7598c589fe
|
Detect other invalid UTF-8 encodings
|
2022-05-31 22:26:09 -04:00 |
|
|
josh
|
a0af8b0d7c
|
Rename Rule to RuleSet
|
2022-05-30 15:49:12 -04:00 |
|
|
josh
|
164a4854fb
|
Update README
|
2022-05-30 15:40:31 -04:00 |
|
|
josh
|
ddadc2008b
|
Rename to propane
|
2022-05-28 20:20:03 -04:00 |
|
|
josh
|
fbd215098b
|
Update license years
|
2022-05-27 21:49:54 -04:00 |
|
|
josh
|
bfe2916165
|
Update bundler
|
2022-05-27 00:15:03 -04:00 |
|
|
josh
|
c9bc4832f4
|
bundle update
|
2022-05-27 00:14:26 -04:00 |
|
|
josh
|
6dfef8573f
|
Fix ERB constructor call for Ruby 3.2 warnings
|
2022-05-27 00:12:40 -04:00 |
|
|
josh
|
f3ed678fe1
|
Store tokens in Hash by name
|
2021-09-27 21:40:12 -04:00 |
|
|
josh
|
280b749e38
|
Track Rule IDs
|
2021-09-27 21:29:44 -04:00 |
|
|
josh
|
d6779aef00
|
Start on Parser#build_tables
|
2021-09-22 23:26:36 -04:00 |
|
|
josh
|
746ec89be8
|
Add test for a rule that can be arrived at from multiple states
|
2021-09-21 21:40:11 -04:00 |
|
|
josh
|
997f34a1e4
|
Keep track of item set in-links
|
2021-09-21 21:32:18 -04:00 |
|
|
josh
|
a2795bb531
|
Keep track of follow item sets by symbol for each item set
|
2021-09-21 17:09:53 -04:00 |
|
|
josh
|
850e639e3a
|
update identical rule spec to use lookahead symbol
|
2021-09-06 20:18:17 -04:00 |
|
|
josh
|
5f7e548fe3
|
Remove Rule::Pattern, Item stores a Rule reference
|
2021-09-06 19:41:29 -04:00 |
|
|
josh
|
bdb10e7afc
|
test duplicate rules
|
2021-09-05 09:50:04 -04:00 |
|
|
josh
|
7bdaf7cdbc
|
Do not create item set following EOF token
|
2021-09-05 07:51:59 -04:00 |
|
|
josh
|
08e3516ad9
|
Add wikipedia LR(0) parser example test
|
2021-09-04 22:33:34 -04:00 |
|
|
josh
|
2c8f3c6e9a
|
Avoid infinite loop with self-referential rules
|
2021-09-04 22:29:10 -04:00 |
|
|
josh
|
9dffa3c41a
|
Recursively build item sets
|
2021-08-29 12:38:44 -04:00 |
|
|
josh
|
ceb7e9ee32
|
Add EOF token to Start rule patterns
|
2021-08-29 11:48:49 -04:00 |
|
|
josh
|
6026bf1514
|
Start building following item sets
|
2021-08-29 09:41:00 -04:00 |
|
|
josh
|
9cc1890ddc
|
One Rule object stores all alternative patterns
|
2021-08-28 10:28:50 -04:00 |
|
|
josh
|
e4f2fffe50
|
add Item#closed_items
|
2021-08-28 09:47:01 -04:00 |
|
|
josh
|
d931bcb513
|
Do not expand rules
|
2021-08-28 09:23:08 -04:00 |
|
|
josh
|
2e16b0bd6e
|
Start on Item and ItemSet
|
2021-08-28 09:02:19 -04:00 |
|
|
josh
|
6ce94e15af
|
Expand rules
|
2021-08-28 08:11:06 -04:00 |
|
|
josh
|
3f92ae46c4
|
Map rule components to Token/Rule references
|
2021-08-22 21:21:41 -04:00 |
|
|
josh
|
00016f16b3
|
Combine Grammar and Generator into top-level Imbecile class
|
2021-08-22 21:04:46 -04:00 |
|
|
josh
|
9273bfccf6
|
Move Token/Rule out of Grammar class
|
2021-08-19 20:00:40 -04:00 |
|
|
josh
|
f295acb593
|
Generator builds a Lexer, not a Lexer::DFA
|
2021-08-19 13:11:12 -04:00 |
|
|
josh
|
51a31317a6
|
Move FA#build_tables to Lexer::DFA
|
2021-08-19 11:55:34 -04:00 |
|
|
josh
|
9459883e74
|
Add Lexer class; Move LexerDFA to Lexer::DFA
|
2021-08-18 17:09:45 -04:00 |
|
|
josh
|
28591907c1
|
Move FA class out of Regex class
|
2021-08-18 17:05:03 -04:00 |
|
|
josh
|
37d6917b49
|
Rework Rule constructor
|
2021-07-27 21:22:46 -04:00 |
|
|
josh
|
2685c05360
|
Change rule syntax
|
2021-07-19 21:55:08 -04:00 |
|
|
josh
|
c0c3353fd7
|
Test lexing empty null string returns EOF
|
2021-07-06 12:06:07 -04:00 |
|
|
josh
|
3158e51059
|
Add length field to LexedToken
|
2021-07-06 11:59:35 -04:00 |
|
|
josh
|
d9e4f64d2e
|
Fix returning TOKEN_EOF when lexing at EOF
|
2021-07-06 11:55:44 -04:00 |
|
|
josh
|
ec2dcf9a72
|
Fix not progressing through input while lexing a token
|
2021-07-06 11:47:33 -04:00 |
|
|
josh
|
578e165e2d
|
Fix off-by-one error in state IDs
|
2021-07-06 11:44:03 -04:00 |
|
|
josh
|
e8df4296cc
|
Begin testing lexer
|
2021-07-06 11:09:39 -04:00 |
|
|
josh
|
230c324209
|
Fix iterating through all transitions in a state
|
2021-07-06 11:09:13 -04:00 |
|
|
josh
|
1271e19b50
|
Test multi-byte code point decoding
|
2021-07-06 11:02:43 -04:00 |
|
|
josh
|
12e11399af
|
Add decoder tests
|
2021-07-06 10:57:06 -04:00 |
|
|
josh
|
24fab8515d
|
Decoder.decode_code_point returns struct with code point and length together
|
2021-07-06 10:50:32 -04:00 |
|
|
josh
|
1dcdd87a28
|
Generate token constants and names to top-level parser class
|
2021-07-06 10:28:35 -04:00 |
|
|
josh
|
8aec7ec0de
|
Lexer class can be used standalone
|
2021-07-06 10:15:07 -04:00 |
|
|
josh
|
c96d55b031
|
Fix class name
|
2021-07-06 10:14:14 -04:00 |
|
|
josh
|
ca7d4862f9
|
Run test executable; build with unit tests
|
2021-07-06 10:03:42 -04:00 |
|
|
josh
|
3c874ae4c1
|
Compile generated parser with a test file
|
2021-07-05 23:05:55 -04:00 |
|
|
josh
|
748c219625
|
Do not return dropped tokens from Lexer.lex_token()
|
2021-07-05 22:53:58 -04:00 |
|