Skip to content

Commit 40d1986

Browse files
mars167mars167
authored
Merge pull request #24 from mars167/feat/enhance-typescript-parser
feat: enhance TypeScript parser to extract arrow functions and export…
2 parents 3346bd0 + 245f01a commit 40d1986

5 files changed

Lines changed: 566 additions & 2 deletions

File tree

‎docs/TYPESCRIPT_PARSER_ENHANCEMENTS.md‎

Lines changed: 208 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,208 @@
1+
# TypeScript Parser Enhancements
2+
3+
## Overview
4+
5+
Enhanced the TypeScript AST parser to extract more symbol types, improving symbol discovery for modern TypeScript/JavaScript patterns.
6+
7+
## Problem
8+
9+
The original parser only extracted:
10+
- `function_declaration` (traditional function declarations)
11+
- `method_definition` (class methods)
12+
- `class_declaration` (classes)
13+
14+
This missed many common patterns in modern codebases:
15+
- Arrow function assignments: `const foo = () => {}`
16+
- Function expression assignments: `const bar = function() {}`
17+
- Exported constants: `export const CONFIG = { ... }`
18+
- Zod schemas: `export const Schema = z.object({ ... })`
19+
- Commander.js commands: `export const cmd = new Command().option(...)`
20+
- Re-exported symbols: `export { foo, bar }`
21+
22+
## Solution
23+
24+
### 1. Added Variable/Constant Symbol Extraction
25+
26+
**Pattern**: `const/let/var name = value`
27+
28+
Extracts symbols for:
29+
- Arrow functions: `const handleSearchFiles = async (input) => { ... }`
30+
- Function expressions: `const helper = function() { ... }`
31+
- Exported constants: `export const SearchFilesSchema = z.object({ ... })`
32+
33+
**Implementation**:
34+
```typescript
35+
else if (n.type === 'lexical_declaration' || n.type === 'variable_declaration') {
36+
for (let i = 0; i < n.namedChildCount; i++) {
37+
const declarator = n.namedChild(i);
38+
if (declarator?.type === 'variable_declarator') {
39+
const nameNode = declarator.childForFieldName('name');
40+
const valueNode = declarator.childForFieldName('value');
41+
42+
if (nameNode && valueNode) {
43+
const isFunction = valueNode.type === 'arrow_function' ||
44+
valueNode.type === 'function' ||
45+
valueNode.type === 'function_expression';
46+
47+
if (isFunction) {
48+
// Extract as function symbol
49+
symbols.push({
50+
name: nameNode.text,
51+
kind: 'function',
52+
...
53+
});
54+
} else if (parent?.type === 'export_statement') {
55+
// Extract exported constants
56+
symbols.push({
57+
name: nameNode.text,
58+
kind: 'variable',
59+
...
60+
});
61+
}
62+
}
63+
}
64+
}
65+
}
66+
```
67+
68+
### 2. Added Export Clause Symbol Extraction
69+
70+
**Pattern**: `export { foo, bar }`
71+
72+
Extracts individual exported names from export clauses.
73+
74+
**Implementation**:
75+
```typescript
76+
else if (n.type === 'export_statement') {
77+
const exportClause = n.childForFieldName('declaration');
78+
if (exportClause?.type === 'export_clause') {
79+
for (let i = 0; i < exportClause.namedChildCount; i++) {
80+
const specifier = exportClause.namedChild(i);
81+
if (specifier?.type === 'export_specifier') {
82+
const nameNode = specifier.childForFieldName('name');
83+
if (nameNode) {
84+
symbols.push({
85+
name: nameNode.text,
86+
kind: 'export',
87+
...
88+
});
89+
}
90+
}
91+
}
92+
}
93+
}
94+
```
95+
96+
### 3. Extended SymbolKind Type
97+
98+
Added new symbol kinds to `src/core/types.ts`:
99+
- `'variable'` - for exported constants and variables
100+
- `'export'` - for re-exported symbols
101+
102+
```typescript
103+
export type SymbolKind =
104+
| 'function'
105+
| 'class'
106+
| 'method'
107+
| 'section'
108+
| 'document'
109+
| 'node'
110+
| 'field'
111+
| 'variable' // NEW
112+
| 'export'; // NEW
113+
```
114+
115+
## Impact
116+
117+
### Before
118+
```typescript
119+
// queryFilesCommand.ts
120+
export const queryFilesCommand = new Command('query-files')
121+
.option('--limit <n>', 'Limit results', '50')
122+
.action(async (pattern, options) => {
123+
await executeHandler('query-files', { pattern, ...options });
124+
});
125+
```
126+
**Result**: 0 symbols extracted ❌
127+
128+
### After
129+
**Result**: 1 symbol extracted ✅
130+
- `queryFilesCommand` (kind: 'variable')
131+
132+
### Before
133+
```typescript
134+
// queryFilesSchemas.ts
135+
export const SearchFilesSchema = z.object({
136+
pattern: z.string(),
137+
limit: z.number()
138+
});
139+
```
140+
**Result**: 0 symbols extracted ❌
141+
142+
### After
143+
**Result**: 1 symbol extracted ✅
144+
- `SearchFilesSchema` (kind: 'variable')
145+
146+
### Before
147+
```typescript
148+
// queryFilesHandlers.ts
149+
export const handleSearchFiles = async (input: SearchFilesInput) => {
150+
return { ok: true };
151+
};
152+
153+
const escapeQuotes = (s: string) => s.replace(/"/g, '\\"');
154+
```
155+
**Result**: 0 symbols extracted ❌
156+
157+
### After
158+
**Result**: 2 symbols extracted ✅
159+
- `handleSearchFiles` (kind: 'function')
160+
- `escapeQuotes` (kind: 'function')
161+
162+
## Test Coverage
163+
164+
Added comprehensive tests in `test/parser-typescript-enhanced.test.ts`:
165+
166+
1. **Arrow function variables** - Extracts arrow functions assigned to constants
167+
2. **Exported constants** - Extracts Zod schemas and configuration objects
168+
3. **Export destructuring** - Handles re-export patterns (note: currently skipped as re-exports without definitions are not extracted)
169+
4. **Commander.js pattern** - Extracts command definitions
170+
5. **Mixed declarations** - Handles combination of traditional functions, arrow functions, classes, and constants
171+
172+
All tests pass ✅
173+
174+
## Benefits
175+
176+
1. **Better Symbol Discovery**: Files like `queryFilesCommand.ts` now have extractable symbols
177+
2. **Improved Graph Queries**: `graph children --as-file` returns meaningful results for more files
178+
3. **Enhanced Search**: Symbol search can find arrow functions and exported constants
179+
4. **Better Context**: LLM review agents get more complete symbol information
180+
181+
## Backward Compatibility
182+
183+
✅ Fully backward compatible
184+
- Existing symbol kinds still work
185+
- New kinds are additive only
186+
- No breaking changes to API or data structures
187+
188+
## Future Enhancements
189+
190+
Potential improvements for future PRs:
191+
192+
1. **Test Function Extraction**: Extract `test('name', () => {})` calls from test files
193+
2. **Interface/Type Extraction**: Extract TypeScript interfaces and type aliases
194+
3. **Enum Extraction**: Extract enum declarations
195+
4. **Namespace Extraction**: Extract namespace declarations
196+
5. **Decorator Extraction**: Extract decorator metadata
197+
198+
## Related Issues
199+
200+
- Fixes symbol extraction for PR #22 files (queryFilesCommand.ts, queryFilesSchemas.ts)
201+
- Improves CodaGraph review quality by providing more complete symbol information
202+
- Addresses empty `graph children` results for modern TypeScript patterns
203+
204+
## Files Changed
205+
206+
- `src/core/types.ts` - Added 'variable' and 'export' to SymbolKind
207+
- `src/core/parser/typescript.ts` - Enhanced extractSymbolsAndRefs() method
208+
- `test/parser-typescript-enhanced.test.ts` - Added comprehensive test suite

‎src/core/parser/typescript.ts‎

Lines changed: 122 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,32 @@ export class TypeScriptAdapter implements LanguageAdapter {
3939
if (n.type === 'call_expression') {
4040
const fn = n.childForFieldName('function') ?? n.namedChild(0);
4141
const callee = extractTsCalleeName(fn);
42-
if (callee) pushRef(refs, callee, 'call', fn ?? n);
42+
if (callee) {
43+
pushRef(refs, callee, 'call', fn ?? n);
44+
45+
// Handle test() and describe() patterns for test files
46+
if (callee === 'test' || callee === 'describe') {
47+
// Extract test name from first argument (usually a string)
48+
const args = n.childForFieldName('arguments');
49+
if (args && args.namedChildCount > 0) {
50+
const firstArg = args.namedChild(0);
51+
if (firstArg?.type === 'string' || firstArg?.type === 'template_string') {
52+
const testName = firstArg.text.replace(/^['"`]|['"`]$/g, '').trim();
53+
if (testName) {
54+
const testSym: SymbolInfo = {
55+
name: testName,
56+
kind: 'test',
57+
startLine: n.startPosition.row + 1,
58+
endLine: n.endPosition.row + 1,
59+
signature: `${callee}("${testName}", ...)`,
60+
container: container,
61+
};
62+
symbols.push(testSym);
63+
}
64+
}
65+
}
66+
}
67+
}
4368
} else if (n.type === 'new_expression') {
4469
const ctor = n.childForFieldName('constructor') ?? n.namedChild(0);
4570
const callee = extractTsCalleeName(ctor);
@@ -82,6 +107,102 @@ export class TypeScriptAdapter implements LanguageAdapter {
82107
symbols.push(classSym);
83108
currentContainer = classSym;
84109
}
110+
} else if (n.type === 'lexical_declaration' || n.type === 'variable_declaration') {
111+
// Handle: const foo = () => {}, const bar = function() {}, const baz = value
112+
for (let i = 0; i < n.namedChildCount; i++) {
113+
const declarator = n.namedChild(i);
114+
if (declarator?.type === 'variable_declarator') {
115+
const nameNode = declarator.childForFieldName('name');
116+
const valueNode = declarator.childForFieldName('value');
117+
118+
if (nameNode && valueNode) {
119+
const isFunction = valueNode.type === 'arrow_function' ||
120+
valueNode.type === 'function' ||
121+
valueNode.type === 'function_expression';
122+
123+
if (isFunction) {
124+
const newSymbol: SymbolInfo = {
125+
name: nameNode.text,
126+
kind: 'function',
127+
startLine: declarator.startPosition.row + 1,
128+
endLine: declarator.endPosition.row + 1,
129+
signature: declarator.text.split('=>')[0].trim() + ' => ...',
130+
container: container,
131+
};
132+
symbols.push(newSymbol);
133+
currentContainer = newSymbol;
134+
} else {
135+
// Also track exported constants/variables
136+
const parent = n.parent;
137+
if (parent?.type === 'export_statement') {
138+
const newSymbol: SymbolInfo = {
139+
name: nameNode.text,
140+
kind: 'variable',
141+
startLine: declarator.startPosition.row + 1,
142+
endLine: declarator.endPosition.row + 1,
143+
signature: declarator.text.split('=')[0].trim(),
144+
container: container,
145+
};
146+
symbols.push(newSymbol);
147+
}
148+
}
149+
}
150+
}
151+
}
152+
} else if (n.type === 'export_statement') {
153+
// Handle: export { foo, bar }
154+
const exportClause = n.childForFieldName('declaration');
155+
if (exportClause?.type === 'export_clause') {
156+
for (let i = 0; i < exportClause.namedChildCount; i++) {
157+
const specifier = exportClause.namedChild(i);
158+
if (specifier?.type === 'export_specifier') {
159+
const nameNode = specifier.childForFieldName('name');
160+
if (nameNode) {
161+
const newSymbol: SymbolInfo = {
162+
name: nameNode.text,
163+
kind: 'export',
164+
startLine: specifier.startPosition.row + 1,
165+
endLine: specifier.endPosition.row + 1,
166+
signature: `export { ${nameNode.text} }`,
167+
container: container,
168+
};
169+
symbols.push(newSymbol);
170+
}
171+
}
172+
}
173+
}
174+
} else if (n.type === 'type_alias_declaration') {
175+
// Handle: type MyType = string | number;
176+
const nameNode = n.childForFieldName('name');
177+
if (nameNode) {
178+
const typeSym: SymbolInfo = {
179+
name: nameNode.text,
180+
kind: 'type',
181+
startLine: n.startPosition.row + 1,
182+
endLine: n.endPosition.row + 1,
183+
signature: `type ${nameNode.text} = ...`,
184+
container: container,
185+
};
186+
symbols.push(typeSym);
187+
}
188+
} else if (n.type === 'interface_declaration') {
189+
// Handle: interface MyInterface { ... }
190+
const nameNode = n.childForFieldName('name');
191+
if (nameNode) {
192+
const head = n.text.split('{')[0].trim();
193+
const heritage = parseHeritage(head);
194+
const interfaceSym: SymbolInfo = {
195+
name: nameNode.text,
196+
kind: 'interface',
197+
startLine: n.startPosition.row + 1,
198+
endLine: n.endPosition.row + 1,
199+
signature: `interface ${nameNode.text}`,
200+
container: container,
201+
extends: heritage.extends,
202+
implements: heritage.implements,
203+
};
204+
symbols.push(interfaceSym);
205+
}
85206
}
86207

87208
for (let i = 0; i < n.childCount; i++) traverse(n.child(i)!, currentContainer);

‎src/core/types.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
export type SymbolKind = 'function' | 'class' | 'method' | 'section' | 'document' | 'node' | 'field';
1+
export type SymbolKind = 'function' | 'class' | 'method' | 'section' | 'document' | 'node' | 'field' | 'variable' | 'export' | 'type' | 'interface' | 'test';
22

33
export interface SymbolInfo {
44
name: string;

0 commit comments

Comments
 (0)