[Phase 1] Task 1.4: Config parser foundation (data model, lexer, 'listen') - #11
Merged
Conversation
Define the structs the parser will fill: ListenSpec (host+port), LocationConfig, ServerConfig, and a non-copyable Config container. Defaults live in constructors (C++98 has no in-class member initialisers for non-static members). Field coverage tracks the mandatory subject requirements (IV.3): interface:port pairs, error pages, body-size limit, per-route methods, redirect, root, autoindex, index, upload store, CGI by extension. Config::load() is declared but not yet implemented; calling it without the parser would be a link error, which is intentional. Related to task: 1.4
Five token kinds (WORD, LBRACE, RBRACE, SEMI, EOF), each carrying line number for error reporting. Eager tokenisation — the whole input is converted into a std::vector<Token> upfront, then walked by the parser via peek()/next(). Config files are kilobytes; the memory cost is negligible and the parser stays a simple cursor. Lexer is "infallible" by design: every byte sequence produces a valid token stream. All real diagnostics are deferred to the parser, which has the directive context needed to give them meaning. Adds a dev-only LEXER_DUMP=<path> env var to main() that prints the token stream of a config file to stderr, then exits. Used to validate the lexer in isolation; remove before submission. Related to task: 1.4
Parser turns the Lexer's token stream into a vector<ServerConfig>.
One method per grammar production (parseFile -> parseServerBlock
-> parseServerDirective), built on two primitives:
- match(kind): consume if next token matches, else leave in place
- expect(kind, context): consume or throw ConfigException
Implements 'listen' for now (bare port or host:port, IPv6
bracketed form tolerated). Other directives produce a warning and
are skipped, matching nginx leniency and respecting the subject's
allowance for extra keys (IV.3).
Unknown-directive recovery tracks brace depth so block-form
directives like `location / { ... }` are skipped as a whole unit;
otherwise an inner ';' would prematurely end the recovery and
mis-pair the closing '}' with the enclosing server block.
Error messages follow the GCC-style "<path>:<line>: ..." format,
so editors can jump straight to the offending line.
Config::load() wires file read -> Lexer -> Parser. A dev-only
CONFIG_DUMP=<path> env var in main() drives it end-to-end.
Related to task: 1.4
There was a problem hiding this comment.
Pull request overview
This PR lays the groundwork for an nginx-style configuration parser by introducing the core configuration data model plus a lexer and recursive-descent parser that currently implements only the listen directive (unknown directives are warned and skipped).
Changes:
- Added config data structures (
Config,ServerConfig,LocationConfig,ListenSpec, etc.) with C++98 constructor defaults. - Implemented an eager-tokenizing lexer and a parser that can parse
server { ... }blocks andlistendirectives with line-aware diagnostics. - Added dev-only
LEXER_DUMP/CONFIG_DUMPpaths inmain()to dump tokens or parsed listen entries.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| src/main.cpp | Adds dev-only env-var driven lexer/config dump utilities before server startup. |
| src/config/parser.cpp | Implements recursive-descent parser, listen parsing, and unknown-directive recovery. |
| src/config/lexer.cpp | Implements eager lexer producing WORD/{/}/;/EOF tokens with line numbers. |
| src/config/config.cpp | Implements Config constructors/defaults and Config::load() lexer→parser glue. |
| includes/webserv.hpp | Exposes lexer types via central project header. |
| includes/parser.hpp | Declares the Parser API and parsing helpers. |
| includes/lexer.hpp | Declares TokenKind, Token, and the Lexer API. |
| includes/config.hpp | Defines the config data model and Config interface. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+70
to
+79
| while (true) { | ||
| const Token& t = lex_.peek(); | ||
| if (t.kind == TOK_EOF) return; | ||
|
|
||
| if (t.kind == TOK_RBRACE) { | ||
| if (depth == 0) return; // not ours; leave for caller | ||
| lex_.next(); | ||
| --depth; | ||
| if (depth == 0) return; // matched the body's '{' | ||
| continue; |
Comment on lines
+71
to
+75
| static std::string readFile(const std::string& path) { | ||
| std::ifstream in(path.c_str()); | ||
| if (!in) | ||
| throw ConfigException("cannot open config file '" + path + "'"); | ||
| std::stringstream ss; |
Completes the server-level leaf directives. Three patterns introduced: Trivial single-value: server_name, root. Read WORD, expect SEMI, assign to the server config field. Parsed-value: client_max_body_size. Accepts a bare byte count or a number with K/M/G suffix (case-insensitive). Implemented in parseSize() with overflow check via std::numeric_limits. Variable-arity: error_page CODE [CODE ...] PATH;. Read WORDs until the closing SEMI, then the last arg is the path and every preceding arg is an HTTP status code (100..599). Same path may be mapped to multiple codes (e.g. 500 502 503 504 -> /50x.html). Refactor: parsePort and the new parseStatusCode share the same "WORD -> integer in [lo, hi]" logic. Factored into parseIntInRange now that we have three call sites (port, status code, parseSize uses it on the digit prefix). CONFIG_DUMP diagnostic in main() now prints the new fields as well. Related to task: 1.4
Adds the location block grammar:
location_block ::= "location" WORD "{" location_directive* "}"
location_directive ::= one of seven supported directives, else warn+skip
parseLocation reads the path WORD, parseLocationBlock walks the body
(structural twin of parseServerBlock), parseLocationDirective is the
dispatcher.
Seven inner directives implemented:
- allowed_methods METHOD [METHOD ...]; -- validated against {GET,POST,DELETE}
- index FILE; -- single WORD
- autoindex on|off; -- enum-style; rejects anything else
- return CODE TARGET; -- code parsed via parseStatusCode
- root PATH; -- overrides server-level root
- upload_store PATH; -- per subject IV.3
- cgi EXT INTERPRETER; -- extension must start with '.'
CONFIG_DUMP diagnostic now prints location entries with all their
fields so the test driver shows the full parsed tree.
After this step, config/default.conf parses to a complete
ServerConfig. The only remaining warnings are 'fastcgi_pass' (out of
scope — we use simple CGI fork/exec, not FastCGI) and 'include'
(Step 6).
Related to task: 1.4
`include FILE;` inside a server block reads FILE and parses its server-level directives directly into the enclosing ServerConfig, as if its contents had been inlined. Path resolution: relative paths are joined with the directory of the including file (not CWD). nginx-compatible and lets configs move as a unit. Cycle detection: a std::set<std::string> tracks files currently being parsed. The top-level Parser owns the set; sub-Parsers share it via pointer. An include that resolves to a file already in the set throws ConfigException; the entry is removed once the include completes so the same file can be included from disjoint subtrees. Scope: only server-scope include is supported. File-scope (mixing server blocks across files) and location-scope can be added later with analogous sub-driver loops. config/default.conf now parses to a complete configuration with no warnings except 'fastcgi_pass' (FastCGI is a different protocol; out of scope — we use simple CGI fork/exec). Note on build: the Makefile does not track header dependencies, so adding members to Parser broke incremental rebuilds with a silent ABI mismatch / segfault. Always run `make re` after a header change until -MMD -MP is added (Phase 4 polish). Related to task: 1.4
Adds Config::validate(), called at the end of Config::load() after
the parser succeeds. Enforces three rules:
1. At least one server block must exist.
2. Each server must declare at least one 'listen' directive.
3. No two servers may share the same (host, port). Per subject
IV.3 virtual hosting is out of scope, so duplicates are a
config error rather than a Host-header routing opportunity.
Deferred checks (filesystem existence of root/cgi paths, redirect
target reachability) are intentionally left to runtime — files
may legitimately appear after startup, and parse-time fs checks
would make configs less portable.
Error messages include the offending server index and, for
duplicates, the index of the conflicting prior server.
This completes Task 1.4.
Related to task: 1.4
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Task
Description
Foundation for the nginx-style config parser, split into three commits:
ListenSpec,LocationConfig,ServerConfig,Config). Fields cover the subject IV.3 requirements; defaults live in constructors (C++98).WORD,LBRACE,RBRACE,SEMI,EOF), eager tokenisation, every token carries its line number for error reporting.expect/matchprimitives,parseFile → parseServerBlock → parseServerDirectiverecursion. Only thelistendirective is implemented for now; everything else warns and is skipped (nginx-style, subject IV.3 allows extra keys). Recovery tracks brace depth so block-form unknown directives skip cleanly.Error messages use GCC-style
path:line: messageso editors can jump.Testing
make reclean with-Wall -Wextra -Werror -std=c++98config/default.confparses (onlylistenconsumed, rest warned);, missing{, unclosed block, bad port (letters / zero / >65535), empty host, garbage at top level, nonexistent fileTwo dev-only env vars added to
main()for iteration:LEXER_DUMP=<path>andCONFIG_DUMP=<path>. To be removed in Phase 4 cleanup.Not in this PR
Remaining server-level directives (
server_name,root,client_max_body_size,error_page),locationblocks,include, the validation pass, and wiring intomain()to replace the hardcoded port. Those land in follow-up PRs on top of this foundation.