Commit 97de508
authored
feat: PostgreSQL database migration with Drizzle ORM (#165)
* feat: add PostgreSQL database with Drizzle ORM for patterns, use cases, and application patterns
- Add Drizzle ORM and postgres dependencies
- Create database schema for application_patterns, effect_patterns, jobs, and relations
- Implement repository layer with CRUD and search operations
- Add migration script to parse existing JSON/MDX/MD files
- Update toolkit to support both file-based and database-based storage
- Add integration tests for all repositories
- Configure drizzle.config.ts for migrations
Database tables:
- application_patterns: High-level pattern categories
- effect_patterns: Concrete code examples
- jobs: Jobs-to-be-Done entries
- pattern_jobs: Many-to-many pattern-job relationships
- pattern_relations: Related patterns linking
New scripts:
- bun run db:generate - Generate migrations
- bun run db:push - Push schema to database
- bun run db:migrate - Run data migration from files
- bun run db:verify - Verify migration results
- bun run db:studio - Open Drizzle Studio
* feat: add database service layer and update MCP server to use database
- Create DatabaseService with Effect.Service wrapper for repositories
- Add high-level database operations (searchEffectPatterns, findEffectPatternBySlug)
- Update MCP server PatternsService to use database instead of file loading
- Convert database EffectPattern types to legacy Pattern types for compatibility
- Add migration testing guide
Database service provides:
- DatabaseLayer for dependency injection
- Effect-based repository access
- Type conversion from DB schema to legacy Pattern schema
- Error handling and logging integration
* feat: update publishing pipeline to use database
- Replace file-based loading with database queries
- Load application patterns and effect patterns from PostgreSQL
- Find actual file paths by searching filesystem
- Maintain backward compatibility with existing file structure
- Improve path resolution accuracy
* feat: update CLI tools and documentation for database migration
CLI Updates:
- Update search, list, and show commands to use database
- Replace JSON file loading with database queries
- Add proper error handling and database connection management
- Support related patterns query from database
Documentation Updates:
- Update DATA_MODEL.md to reflect PostgreSQL as primary source
- Add database schema documentation
- Update ARCHITECTURE.md with database technologies
- Add database configuration section
- Update package descriptions to mention database
- Add migration guide references
* feat: add comprehensive database testing utilities and guides
Testing Infrastructure:
- Add test-db.ts - Comprehensive database test runner
- Add test-db-quick.ts - Quick database connectivity test
- Add db-helpers.ts - Test utilities (setup, cleanup, seeding)
- Update repositories.test.ts to use test helpers
Documentation:
- Add DATABASE_TESTING.md - Complete testing guide
- Update MIGRATION_TESTING.md with testing steps
Scripts:
- bun run test:db - Run full database test suite
- bun run test:db:quick - Quick connectivity test
- bun run test:db:repositories - Run repository integration tests
* feat: add entity locking mechanism for validated patterns
- Add validated and validatedAt fields to application_patterns, effect_patterns, and jobs tables
- Implement lock/unlock methods in all repositories
- Add locked error types (EffectPatternLockedError, ApplicationPatternLockedError, JobLockedError)
- Enforce read-only behavior for locked entities in update/delete/upsert operations
- Add lock and unlock commands to ep-admin CLI
- Fix postgres import in database client (use default import instead of namespace)
- Fix generate.ts to group patterns by applicationPatternId from database
- Generate database migration for locking fields
- Export locked error types from toolkit package
* feat: update pipeline state and database services with latest changes1 parent 6ccf5e6 commit 97de508
71 files changed
Lines changed: 10500 additions & 1477 deletions
File tree
- content/published/rules
- cursor
- windsurf
- docs
- packages
- cli/src
- pipeline-state/src
- toolkit
- dist
- schemas
- src
- __tests__
- db
- migrations
- meta
- schema
- repositories
- services
- scripts
- publish
- services/mcp-server
- app/api/patterns
- src/server
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Large diffs are not rendered by default.
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
Lines changed: 166 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
Lines changed: 92 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
0 commit comments