96 Commits

Author SHA1 Message Date
779ba052c1 Use P_SUCCESS instead of P_TOKEN 2023-07-11 15:43:15 -04:00
eee6513384 Remove outer namespacing class 2023-07-10 22:54:33 -04:00
78ce7fb77a Replace 'ref' arguments with plain pointers 2023-07-10 22:40:03 -04:00
ad768711ff Use size_t to hold runtime object IDs and INVALID_ID 2023-07-10 22:32:33 -04:00
a2338a8fca Add ShiftID 2023-07-10 22:28:48 -04:00
612eb12545 Add RuleID 2023-07-10 22:27:53 -04:00
ef7488747c Add StateID 2023-07-10 22:25:28 -04:00
950fafc872 Add SymbolID 2023-07-10 22:17:54 -04:00
9dbae78724 Add ReduceID type 2023-07-10 21:55:49 -04:00
f973e9dc2c Rename lexer and parser State structs to avoid conflicts 2023-07-10 11:21:45 -04:00
1c50d37a3e Add Position struct to track text positions 2023-07-10 11:18:57 -04:00
80ac6c17f0 Replace Token struct with integer type
Replace _TOKEN_COUNT with INVALID_TOKEN_ID.
2023-07-09 22:35:02 -04:00
6327bd1e96 Use plain integer type for UserCodeID 2023-07-08 17:01:54 -04:00
36ad6e0d1b Build Lexer and Parser tables upon construction 2023-07-08 16:52:58 -04:00
ffd02c16d1 Add StateID; remove TransitionResult 2023-07-08 16:46:38 -04:00
6daca1e73f Add INVALID_ID 2023-07-08 10:49:56 -04:00
ecef933255 Consolidate return codes 2023-07-08 09:23:31 -04:00
0232b204c6 Return integer result code from Parser.parse() 2023-07-08 08:52:21 -04:00
c88338698a Store whether a state accepts, not whether it drops 2023-07-08 08:29:33 -04:00
8a377b4950 Just return integer result code from Lexer.lex_token() 2023-07-08 08:08:36 -04:00
0d0da49cd5 Just return integer result code from Lexer.find_longest_match() 2023-07-05 17:07:48 -04:00
d56cc2deeb Just return integer result code from Decoder.decode_code_point() 2023-07-05 16:54:52 -04:00
9e33a32930 Store code_point_length in a ubyte 2023-07-05 16:45:03 -04:00
02d99082b2 Allow switching on Decoder.Result 2023-07-04 12:49:54 -04:00
9895733a05 Switch unit tests from gdc to ldc2 2023-06-29 19:16:36 -04:00
11348ca351 Add FindLongestMatchResult 2023-06-04 21:14:07 -04:00
6b0fb4cb12 Add return type for transition() 2023-06-03 20:38:42 -04:00
b78827746a Return a Token object from Lexer.user_code() 2023-03-17 20:24:48 -04:00
ce80e00e60 Remove TOKEN_DECODE_ERROR 2023-03-17 20:01:55 -04:00
01b45df1f9 Remove TOKEN_DROP 2023-03-17 19:58:54 -04:00
ffccc659aa Clean up Lexer.State a bit 2023-03-14 19:38:42 -04:00
e432f62b05 Add UserCodeID struct to represent user code IDs 2023-03-13 21:56:18 -04:00
058945e08b Add Token struct with validity checks 2023-03-13 16:14:50 -04:00
f402315201 Name internally generated tokens with double-underscore 2023-03-13 14:13:13 -04:00
64974cc1e2 Refactor some of Lexer.attempt_lex_token() into find_longest_match() 2023-03-12 21:19:03 -04:00
b92679e0c2 Replace LexedToken with Result struct 2023-03-11 21:16:55 -05:00
1af018b103 MatchInfo: store pointer to the accepting state rather than copying its fields 2023-03-09 20:16:57 -05:00
aabb574fea Use CodePoint type for code point values 2023-03-09 19:10:17 -05:00
30004c571d Use result type query methods on Decoder.Result 2023-03-09 19:04:15 -05:00
b00f53050b Clean up Decoder result handling 2023-03-06 20:32:01 -05:00
2e5083d823 Add explicit base types to the enums 2023-03-06 19:58:24 -05:00
c6ea4f83c2 Store parser values according to the rule/pattern type 2022-11-13 22:20:30 -05:00
8dc27686aa Rename result_type to ptype, parser result to pvalue 2022-11-12 13:29:10 -05:00
bca0a14371 Allow storing a result value for a token from a lexer code block 2022-10-16 21:40:25 -04:00
ca8a360c0e Provide matched text to lexer user code block 2022-10-15 13:37:34 -04:00
623c644e74 Work on real D strings instead of ubyte pointer and length
Also fix a couple UTF-8 decoder bugs!
2022-10-15 13:32:33 -04:00
727c8cd1ea Execute rule user code blocks when reducing the rule 2022-10-13 04:56:50 -04:00
74beaf7ed8 Store final parse result and give access with .result parser property 2022-10-12 21:33:09 -04:00
31970522de Store parse result; add result_type grammar keyword 2022-10-12 20:56:14 -04:00
02be6de48e Add lexer modes and $mode() code expansion 2022-10-09 22:49:01 -04:00