Claude/meshyai integration research 011 c ur m lkk hr53 q trz7s kzkw - #7
Open
WilliamAvHolmberg wants to merge 8 commits into
Open
Conversation
Implements complete MeshyAI Text-to-3D API integration allowing users to generate 3D models on-the-fly using natural language prompts. This is a proof-of-concept implementation with no backend required. Features: - Text-to-3D model generation with natural language prompts - Real-time progress tracking with visual indicators - Persistent local storage using IndexedDB - Model inventory with search, filtering, and management - Multiple art styles (realistic, sculpture, cartoon, low poly, voxel) - PBR material support (metallic, roughness, normal maps) - GLB format output optimized for web/Cesium - Download models as .glb files - Favorites system and auto-tagging - User-provided API key management (localStorage) - Global React overlay accessible from anywhere Architecture: - Service layer: MeshyAPI client + IndexedDB cache - State management: React Context with hooks - UI components: Generator panel, queue viewer, inventory browser - Two-stage generation: Preview (base mesh) → Refine (textures) - Polling-based progress tracking with error handling Files added: - packages/web/src/react/features/model-generator/ (complete feature) - services/: API client, storage, types - hooks/: useModelGenerator - components/: UI components for generator, queue, inventory - context/: GenerationContext for global state - utils/: Helper functions - packages/web/MESHYAI_INTEGRATION.md (comprehensive documentation) Integration: - App.tsx wrapped with GenerationProvider - ModelGeneratorOverlay added as global component - No changes to existing functionality Documentation includes: - Complete setup guide - API flow diagrams - Code examples - Troubleshooting guide - Security considerations - Future enhancement roadmap
MeshyAI's asset download URLs don't include CORS headers, blocking direct browser downloads. This adds a lightweight Express server to proxy API requests and download GLB files server-side. Architecture: Frontend (Vite:5174) → Express (3001) → MeshyAI API/CDN See packages/api/README.md for detailed documentation.
Three critical fixes and new feature: 1. Fix art style validation (API error 400) - Changed 'lowpoly' → 'low-poly' (API requires hyphenated) - Changed 'voxel' → 'pbr' (voxel not supported by API) - Updated estimateGenerationTime to match new styles 2. Fix API key manage button - Button was non-functional (setActiveTab to same tab) - Added showApiKeyManage state to ModelGeneratorPanel - Now shows ApiKeySetup with back button when clicked 3. Add airplane replacement feature - Added replaceModel() method to Vehicle class - Exposed via GameBridge.replaceVehicleModel() - Added to useGameMethod hook - New button in ModelInventory: '✈️ Replace' - Replaces active vehicle with generated GLB model - Shows loading state while replacing Technical changes: - Vehicle.ts: New replaceModel() async method - GameBridge.ts: New replaceVehicleModel() method - useGameMethod.ts: Exposed replaceVehicleModel to React - ModelInventory.tsx: Replace button with loading state - types.ts: Fixed ArtStyle type - modelUtils.ts: Updated time estimates All fixes tested and build passes ✅
Added UI controls to rotate models by 90° increments and adjust scale before replacing the airplane. Each model now has individual rotation/scale settings. Features: - Rotation controls: -90° / +90° buttons with current rotation display - Scale slider: 0.1x to 5.0x with real-time value display - Settings tracked per model (stored in component state) - Applied when replacing airplane model UI Components: - Rotation buttons with ↶ ↷ symbols - Scale slider with min/max labels - Current values displayed prominently - Glass panel styling matching existing design Technical Changes: - Vehicle.ts: Added rotationDegrees parameter to replaceModel() - Converts degrees to radians for modelHeadingOffset - Updates heading when loading new model - GameBridge.ts: Added rotationDegrees parameter to replaceVehicleModel() - useGameMethod.ts: Exposed rotation parameter to React - ModelInventory.tsx: - Added modelSettings state (rotation: 0-359°, scale: 0.1-5.0) - Helper functions: getModelSettings, rotateModel, updateScale - UI controls for rotation and scale per model - Applied settings when replacing airplane User Experience: 1. Adjust rotation (-90° / +90°) and scale (slider) 2. Click '✈️ Replace' to apply model with settings 3. Success message shows applied scale and rotation 4. Each model remembers its individual settings Defaults: rotation: 0°, scale: 1.0x Build passes ✅
- Introduced `getRunWithoutTokens` and `setRunWithoutTokens` functions to manage game behavior when API tokens are not provided. - Updated `initializeGame` to check for the new run without tokens condition. - Modified `saveTokens` to allow saving tokens conditionally and set the run without tokens flag if both tokens are absent. - Adjusted `TokenSetup` component to remove strict token validation, allowing for more flexible token handling. These changes improve the user experience by providing a smoother setup process for the game when tokens are not available.
- Remove all emojis from UI for cleaner, more professional appearance - Restrict art styles to only 'realistic' and 'sculpture' (as per API limitations) - Add automatic tab switching: generate → queue → inventory for seamless UX - Update time estimates to match reduced art style options Components updated: - ModelGeneratorPanel: Auto switch tabs on generate/complete, remove emojis - ModelInventory: Remove emojis from buttons (Download, Replace, Delete) - GenerationQueue: Remove stage icons and empty state emoji - ApiKeySetup: Remove warning emoji
- Add AI 3D Model Generator to features list - Include comprehensive setup instructions for MeshyAI integration - Document proxy server requirement and startup process - Update tech stack and project structure - Add MeshyAI to acknowledgments Users can now generate custom 3D models with text prompts and replace the vehicle model. Requires MeshyAI API key (subscription) and running the Express proxy server in packages/api.
Owner
Author
|
@coderabbitai hi review pelase |
Implement complete image-to-3D workflow alongside existing text-to-3D: - Add image upload with file/URL support (JPG, PNG, max 10MB) - Mode selector UI (Text-to-3D / Image-to-3D) - Multer integration for file uploads in Express proxy - Support for MeshyAI Meshy-6 (latest) and Meshy-4 models - Image preview display in queue and inventory - Mode badges to distinguish generation types Backend changes: - Express: Add /api/meshy/image-to-3d endpoints with multipart/form-data - Convert uploaded images to base64 for MeshyAI API - Add multer dependency for file handling Frontend changes: - Types: Add ImageToModelParams, update GenerationTask/GeneratedModel with mode - MeshyAPI: Add createImageTo3DTask() method, update getTaskStatus() with mode param - Hook: Split generateModel into generateTextTo3D and generateImageTo3D - UI: Mode selector, image upload dropzone, AI model selector, source image display - Queue: Show source image for image-to-3D tasks - Inventory: Mode badge overlay, source image thumbnail Both text-to-3D and image-to-3D workflows fully functional end-to-end.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.