Adding dependencies for python and other project types - #15
Conversation
WalkthroughThe changes expand dependency analysis in the frontend to support Python, PHP, Rust, and Go repositories in addition to Node. This involves updating type definitions, refactoring dependency-fetching logic to support multiple languages, updating UI text to reflect broader support, and adjusting spacing in a folder structure component. The Changes
Sequence Diagram(s)sequenceDiagram
participant UI
participant useFetchDependencies
participant service
participant RepoAPI
participant ExternalRegistry
UI->>useFetchDependencies: Request dependencies for repository
useFetchDependencies->>service: fetchPackagesSummary({ projectType, repositoryDetails })
alt Node project
service->>RepoAPI: Fetch package.json, package-lock.json
service->>ExternalRegistry: Fetch NPM metadata
else Python project
service->>RepoAPI: Fetch requirements.txt, pyproject.toml
service->>ExternalRegistry: Fetch PyPI metadata
else PHP project
service->>RepoAPI: Fetch composer.json
service->>ExternalRegistry: Fetch Packagist metadata
else Rust project
service->>RepoAPI: Fetch Cargo.toml
service->>ExternalRegistry: Fetch crates.io metadata
else Go project
service->>RepoAPI: Fetch go.mod
service->>ExternalRegistry: Fetch Go proxy metadata
end
service-->>useFetchDependencies: Return summarized dependencies
useFetchDependencies-->>UI: Update with dependency data
Poem
✨ Finishing Touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (3)
frontend/src/features/repositories/components/Dependencies/service.ts (3)
255-255: Improve organization extraction logic.Extracting organization from email by splitting on "@" and taking the domain is fragile and may not accurately represent the organization.
Consider using the package maintainer information or homepage URL for more reliable organization data:
- organization: info?.author_email?.split("@")[1]?.split(".")[0], // Extract domain from email + organization: info?.maintainer || info?.author || null, // Use maintainer/author as organization
265-286: Version scoring could be more robust.The current implementation might fail for non-standard version formats (e.g., "1.0.0-alpha", "2.0.0rc1"). Consider using a proper semantic versioning library or adding validation.
Do you want me to suggest a more robust version comparison implementation that handles semantic versioning properly?
420-423: Consider more robust go.mod parsing.The current implementation uses simple string operations to parse go.mod, which might not handle all valid formats (e.g., replace directives, multi-line entries).
Consider using a proper go.mod parser or more sophisticated regex patterns to handle edge cases.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
frontend/package-lock.jsonis excluded by!**/package-lock.json
📒 Files selected for processing (7)
frontend/package.json(1 hunks)frontend/src/features/repositories/components/Dependencies/Dependencies.tsx(1 hunks)frontend/src/features/repositories/components/Dependencies/service.ts(3 hunks)frontend/src/features/repositories/components/Dependencies/useFetchDependencies.tsx(1 hunks)frontend/src/features/repositories/components/RepositoriesTable/types.ts(1 hunks)frontend/src/features/repositories/components/RepositoryCards/FolderStructureDisplay.tsx(2 hunks)frontend/src/types/repository.ts(1 hunks)
🧰 Additional context used
🧬 Code Graph Analysis (3)
frontend/src/features/repositories/components/RepositoriesTable/types.ts (1)
frontend/src/types/repository.ts (1)
ProjectType(43-43)
frontend/src/features/repositories/components/Dependencies/useFetchDependencies.tsx (1)
frontend/src/features/repositories/components/Dependencies/service.ts (1)
fetchPackagesSummary(463-489)
frontend/src/features/repositories/components/Dependencies/service.ts (3)
frontend/src/types/repository.ts (2)
RepositoryInfo(15-41)ProjectType(43-43)frontend/src/features/repositories/components/Dependencies/types.ts (1)
PackageDetails(1-10)frontend/src/features/repositories/services/repositories.ts (1)
fetchFileContent(267-285)
🔇 Additional comments (9)
frontend/package.json (1)
46-46:Details
❓ Verification inconclusive
Verify the TOML package version and security.
The addition of the
tomlpackage supports the new multi-language dependency parsing functionality. Please ensure that version3.0.0is the latest stable version and check for any security vulnerabilities.
🏁 Script executed:
#!/bin/bash # Description: Verify the TOML package version and check for security issues # Check latest version of toml package npm view toml version npm view toml versions --json | jq '.[-5:]' # Check for security advisories npm audit --audit-level moderate --package-lock-only toml || trueLength of output: 549
Confirm toml@3.0.0 version and security scan
npm view toml versionconfirms 3.0.0 is the latest stable release.- A proper
npm auditrequires a lockfile. Please run:to verify there are no known vulnerabilities.npm install toml@3.0.0 --package-lock-only npm audit --audit-level moderate- Optionally, cross-check with the NPM Advisory Database or GitHub Dependabot alerts before merging.
frontend/src/types/repository.ts (2)
37-37: Excellent type safety improvement!Changing from a generic
stringto the specificProjectTypeunion type enhances type safety and makes the supported project types explicit.
43-43: Well-structured ProjectType definition.The union type covers all the major project types mentioned in the PR objectives plus a sensible "unknown" fallback. The inclusion of "c++" alongside the other languages provides good extensibility.
frontend/src/features/repositories/components/RepositoriesTable/types.ts (2)
1-1: Good import addition for type consistency.Importing the
ProjectTypefrom the central repository types ensures consistency across the codebase.
8-8: Consistent type safety improvement.Using the specific
ProjectTypeinstead of a generic string maintains type consistency with the main repository interface and provides better type safety.frontend/src/features/repositories/components/RepositoryCards/FolderStructureDisplay.tsx (1)
47-47: UI spacing adjustments look good.The changes from
py-1topb-1andmr-1tomr-0create a more compact folder tree display by reducing vertical spacing between items and tightening the button margin. These are reasonable UI refinements.Also applies to: 51-51, 72-72
frontend/src/features/repositories/components/Dependencies/useFetchDependencies.tsx (1)
17-20: Clean refactoring to support multi-language dependencies.The hook now properly delegates language-specific dependency fetching to the service layer by passing
projectType. This improves separation of concerns and enables the multi-language support.frontend/src/features/repositories/components/Dependencies/service.ts (2)
79-152: Node.js dependency fetching looks good.The refactored function maintains the original functionality with proper error handling and concurrent request limiting using
pLimit(10).
463-489: Excellent type-safe dispatcher implementation.The switch statement with exhaustive checking ensures all project types are handled and will cause compile-time errors if new types are added without implementation.
| > | ||
| {dependenciesCheckIsAvailable === false ? ( | ||
| <UnderConstruction content="For the time being, this feature is only available for node repositories" /> | ||
| <UnderConstruction content="For the time being, this feature is only available for Node, Python or PHP repositories" /> |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Update message to reflect all supported project types.
The message only mentions "Node, Python or PHP repositories" but the ProjectType definition includes additional languages like "rust", "go", and "c++". This inconsistency could confuse users about what's actually supported.
Consider updating the message to include all supported types:
-<UnderConstruction content="For the time being, this feature is only available for Node, Python or PHP repositories" />
+<UnderConstruction content="For the time being, this feature is only available for Node, Python, PHP, Rust, Go, and C++ repositories" />Alternatively, use a more generic message to avoid maintaining a list:
-<UnderConstruction content="For the time being, this feature is only available for Node, Python or PHP repositories" />
+<UnderConstruction content="This feature is currently available for supported project types only" />📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| <UnderConstruction content="For the time being, this feature is only available for Node, Python or PHP repositories" /> | |
| <UnderConstruction content="For the time being, this feature is only available for Node, Python, PHP, Rust, Go, and C++ repositories" /> |
🤖 Prompt for AI Agents
In frontend/src/features/repositories/components/Dependencies/Dependencies.tsx
at line 36, the message in the UnderConstruction component only lists Node,
Python, and PHP as supported project types, but the ProjectType definition
includes additional languages like Rust, Go, and C++. Update the message to
either list all supported project types accurately or replace it with a more
generic statement that covers all supported languages without enumerating them,
ensuring consistency and avoiding confusion.
| const fetchPhpProjectDependencies = async ({ | ||
| repositoryDetails, | ||
| }: { | ||
| repositoryDetails: RepositoryInfo; | ||
| }): Promise<PackageDetails[]> => { | ||
| try { | ||
| const composerContent = await fetchFileContent({ | ||
| repositoryDetails, | ||
| path: "composer.json", | ||
| }); | ||
|
|
||
| if (!composerContent) return []; | ||
|
|
||
| const composerJson = JSON.parse(composerContent) as { | ||
| require?: Record<string, string>; | ||
| "require-dev"?: Record<string, string>; | ||
| }; | ||
| const packages: PackageDetails[] = []; | ||
|
|
||
| const allDependencies = { | ||
| ...(composerJson.require || {}), | ||
| ...(composerJson["require-dev"] || {}), | ||
| }; | ||
|
|
||
| for (const [name, version] of Object.entries(allDependencies)) { | ||
| try { | ||
| const packagistResponse = await axios.get(`https://repo.packagist.org/p2/${name}.json`); | ||
| const packageData = packagistResponse.data.packages[name][0]; | ||
|
|
||
| packages.push({ | ||
| name, | ||
| usedVersion: version.toString().replace(/^[\^~]/, ""), | ||
| latestVersion: packageData.version, | ||
| author: packageData.authors?.[0]?.name || null, | ||
| organization: name.split("/")[0] || null, | ||
| lastUpdate: packageData.time || null, | ||
| description: packageData.description || null, | ||
| dependencyScore: calculateDependencyScore(version.toString(), packageData.version), | ||
| }); | ||
| } catch (error) { | ||
| console.error(`Failed to fetch Packagist data for ${name}:`, error); | ||
| packages.push({ | ||
| name, | ||
| usedVersion: version.toString().replace(/^[\^~]/, ""), | ||
| latestVersion: null, | ||
| author: null, | ||
| organization: null, | ||
| lastUpdate: null, | ||
| description: null, | ||
| dependencyScore: null, | ||
| }); | ||
| } | ||
| } | ||
|
|
||
| return packages; | ||
| } catch (error) { | ||
| console.error("Error processing PHP project dependencies:", error); | ||
| return []; | ||
| } | ||
| }; |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Add request throttling to PHP, Rust, and Go implementations.
All three language implementations lack concurrent request limiting, which could cause rate limiting issues or browser errors for projects with many dependencies.
Apply the same pLimit pattern used in the Node.js implementation to prevent overwhelming external APIs and improve reliability.
Also applies to: 349-404, 406-461
🤖 Prompt for AI Agents
In frontend/src/features/repositories/components/Dependencies/service.ts around
lines 288 to 347, the fetchPhpProjectDependencies function makes concurrent HTTP
requests to the Packagist API without any concurrency control, risking rate
limiting or browser errors for projects with many dependencies. To fix this,
import and use the pLimit library to limit the number of simultaneous axios.get
requests. Wrap the axios.get call inside a pLimit-limited function and execute
all requests through this limiter to ensure only a fixed number of concurrent
requests run at once. Apply the same pLimit concurrency control pattern to the
Rust and Go dependency fetching functions in lines 349-404 and 406-461
respectively.
| const fetchPythonProjectDependencies = async ({ | ||
| repositoryDetails, | ||
| }: { | ||
| repositoryDetails: RepositoryInfo; | ||
| }): Promise<PackageDetails[]> => { | ||
| try { | ||
| // 1. Fetch dependency files | ||
| const requirementsFile = await fetchFileContent({ | ||
| repositoryDetails, | ||
| path: "requirements.txt", | ||
| }); | ||
| const pyProjectFile = await fetchFileContent({ repositoryDetails, path: "pyproject.toml" }); | ||
|
|
||
| const packages: PackageDetails[] = []; | ||
|
|
||
| // 2. Parse requirements.txt if exists | ||
| if (requirementsFile) { | ||
| const requirements = requirementsFile | ||
| .split("\n") | ||
| .filter((line) => line.trim() && !line.trim().startsWith("#")) | ||
| .map((line) => { | ||
| // Extract package name and version (handling cases like "package==1.0.0") | ||
| const match = line.trim().match(/^([a-zA-Z0-9_-]+)([=<>~!]=?.*)?$/); | ||
| return match | ||
| ? { name: match[1], version: match[2]?.replace(/^[=<>~!]=?/, "") || null } | ||
| : null; | ||
| }) | ||
| .filter(Boolean); | ||
|
|
||
| for (const req of requirements) { | ||
| if (req) { | ||
| const { name, version } = req; | ||
| const pkgInfo = await getPyPIMetadata(name); | ||
| if (!pkgInfo) continue; // Skip if package metadata not found | ||
|
|
||
| packages.push({ | ||
| name, | ||
| usedVersion: version, | ||
| latestVersion: pkgInfo?.version || null, | ||
| author: pkgInfo?.author || null, | ||
| organization: pkgInfo?.organization || null, | ||
| lastUpdate: pkgInfo?.last_update || null, | ||
| description: pkgInfo?.description || null, | ||
| dependencyScore: calculateDependencyScore(version, pkgInfo?.version), | ||
| }); | ||
| } | ||
| } | ||
| } | ||
|
|
||
| // 3. Parse pyproject.toml if exists | ||
| if (pyProjectFile) { | ||
| try { | ||
| const pyProject = parse(pyProjectFile); | ||
| const deps = [ | ||
| ...(pyProject.tool?.poetry?.dependencies || []), | ||
| ...(pyProject.project?.dependencies || []), | ||
| ]; | ||
|
|
||
| for (const [name, versionObj] of Object.entries(deps)) { | ||
| if (typeof name !== "string") continue; | ||
|
|
||
| const version = typeof versionObj === "string" ? versionObj : versionObj?.version; | ||
| if (!packages.some((p) => p.name === name)) { | ||
| const pkgInfo = await getPyPIMetadata(name); | ||
| packages.push({ | ||
| name, | ||
| usedVersion: version, | ||
| latestVersion: pkgInfo?.version || null, | ||
| author: pkgInfo?.author || null, | ||
| organization: pkgInfo?.organization || null, | ||
| lastUpdate: pkgInfo?.last_update || null, | ||
| description: pkgInfo?.description || null, | ||
| dependencyScore: calculateDependencyScore(version, pkgInfo?.version), | ||
| }); | ||
| } | ||
| } | ||
| } catch (tomlError) { | ||
| console.error("Error parsing pyproject.toml:", tomlError); | ||
| } | ||
| } | ||
|
|
||
| return packages; | ||
| } catch (error) { | ||
| console.error("Error fetching dependencies:", error); | ||
| return []; | ||
| } | ||
| }; |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Add request throttling for Python dependencies.
The function correctly handles both requirements.txt and pyproject.toml, but unlike the Node.js implementation, it lacks concurrent request limiting. This could cause issues with large dependency lists.
Add request throttling similar to the Node.js implementation:
const fetchPythonProjectDependencies = async ({
repositoryDetails,
}: {
repositoryDetails: RepositoryInfo;
}): Promise<PackageDetails[]> => {
try {
+ // This limits the number of concurrent requests to avoid rate limiting
+ const limit = pLimit(10);
// 1. Fetch dependency filesThen wrap the PyPI metadata calls with the limiter:
- const pkgInfo = await getPyPIMetadata(name);
+ const pkgInfo = await limit(() => getPyPIMetadata(name));Committable suggestion skipped: line range outside the PR's diff.
🤖 Prompt for AI Agents
In frontend/src/features/repositories/components/Dependencies/service.ts between
lines 154 and 240, the fetchPythonProjectDependencies function makes multiple
concurrent calls to getPyPIMetadata without any concurrency control, which can
overwhelm the PyPI API for large dependency lists. To fix this, introduce a
request throttling mechanism similar to the Node.js implementation by creating a
limiter (e.g., using p-limit or a similar concurrency control library) and wrap
all calls to getPyPIMetadata with this limiter to ensure only a limited number
of concurrent requests are made.
1 participant
Summary by CodeRabbit
New Features
Improvements
Style