Skip to content

D11225687/Text2SQL

Folders and files

NameName
Last commit message
Last commit date

Latest commit

ย 

History

2 Commits
ย 
ย 

Repository files navigation

ๅพž้›ถๆ‰“้€  Text2SQL ๅผ•ๆ“Ž:ๅคš้›ฒ LLM ๅˆ‡ๆ›็ญ–็•ฅ่ˆ‡ๅทฅ็จ‹ๅฏฆๆˆฐ

ๅพž .NET ๅ‚ณ็ตฑๆžถๆง‹ๅˆฐ Python / Cloud-Native AI ๅนณๅฐ,ๆˆ‘ๅ€‘ๅœจ Navi 2.0 ็”จ 14 ๅ€‹ๆœˆๆ™‚้–“,่ฎ“ใ€Œ่‡ช็„ถ่ชž่จ€ๆŸฅ่ณ‡ๆ–™ๅบซใ€้€™ไปถไบ‹็œŸ็š„่ƒฝ่ท‘ๅœจ productionใ€‚้€™็ฏ‡ๅˆ†ไบซๅพžๆžถๆง‹่จญ่จˆใ€Prompt ๅทฅ็จ‹ใ€ๅคš้›ฒๅˆ‡ๆ›ๅˆฐ่ฉ•ไผฐๆ–นๆณ•็š„ๅฎŒๆ•ดๅฏฆๆˆฐๅฟƒๅพ—ใ€‚


ๅ‰่จ€:็‚บไป€้บผ่‡ชๅทฑๅš Text2SQL?

ๅœจไผๆฅญๅ…ง้ƒจ,่ณ‡ๆ–™ๅˆ†ๆž็š„ๆœ€ๅคง็—›้ปžไธๆ˜ฏใ€Œ่ณ‡ๆ–™ไธๅค ใ€,่€Œๆ˜ฏใ€Œๆœƒ SQL ็š„ไบบๅคชๅฐ‘ใ€ใ€‚็”ขๅ“็ซฏใ€ๆฅญๅ‹™็ซฏใ€ๅฎขๆœ็ซฏๆฏๅคฉๆœ‰ๅคง้‡็š„ใ€Œๆˆ‘ๆƒณ็Ÿฅ้“ๆŸ X ็š„่ณ‡ๆ–™ใ€้œ€ๆฑ‚,ๆœ€ๅพŒ้ƒฝ่ฎŠๆˆๅทฅ็จ‹ๅธซๆŽ’้šŠๆ”น ad-hoc queryใ€‚

ๆˆ‘ๅ€‘ๅœจ่“‹ไบžๅš Navi 2.0 ๆ™‚,ๆ ธๅฟƒ็›ฎๆจ™ๅฐฑๆ˜ฏ่ฎ“ไปปไฝ•ไธ€ๅ€‹ๆฅญๅ‹™ๅŒไบ‹้ƒฝ่ƒฝ็”จไธญๆ–‡ๅ•ๅ•้กŒ,็ณป็ตฑ่‡ชๅทฑ็”Ÿ SQLใ€ๆŸฅ่ณ‡ๆ–™ๅบซใ€ๅ›žๆ‡‰็ตๆžœใ€‚

ๅธ‚้ขไธŠๆœ‰ๅพˆๅคš Text2SQL ่งฃๆฑบๆ–นๆกˆ(LangChain SQLDatabaseChainใ€Vannaใ€DataHerald ็ญ‰),ไฝ†่ท‘ไธ‹ไพ†้ƒฝ้‡ๅˆฐๅŒๆจฃ็š„ๅ•้กŒ:

  1. ่ทจ่ณ‡ๆ–™ๅบซๆ”ฏๆดไธไธ€่‡ดโ€”โ€”MySQL ่ทŸ MSSQL ็š„ๆ–น่จ€ใ€ไฟ็•™ๅญ—ใ€JOIN ่ชžๆณ•ไธๅŒ
  2. Schema ๅคชๅคงๆœƒ token ็ˆ†็‚ธโ€”โ€”็›ดๆŽฅๆŠŠ schema ๅ…จๅกž็ตฆ LLM,ๅ‹•่ผ’ 50K+ token
  3. ๆบ–็ขบ็އไธ็ฉฉโ€”โ€”ๅŒไธ€ๅ€‹ๅ•้กŒ็ฌฌไธ€ๆฌก็ญ”ๅฐ,็ฌฌไบŒๆฌก็ญ”้Œฏ
  4. ๅ–ฎไธ€ LLM ้ขจ้šชโ€”โ€”็ถๆญป OpenAI ็ญ‰ๆ–ผๆŠŠๆˆๆœฌ่ˆ‡ๅฏ็”จๆ€ง้ƒฝไบค็ตฆๅ–ฎไธ€ไพ›ๆ‡‰ๅ•†

ๆ–ผๆ˜ฏๆˆ‘ๅ€‘ๆฑบๅฎš่‡ชๅทฑๅšใ€‚


็ณป็ตฑๆžถๆง‹ๆฆ‚่ฆฝ

                    โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
                    โ”‚        User Natural Language         โ”‚
                    โ”‚   "ไธŠๅ€‹ๆœˆ้Šทๅ”ฎๅ‰ไธ‰ๅ็š„ๅ•†ๅ“ๆ˜ฏไป€้บผ?"      โ”‚
                    โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
                                      โ”‚
                                      โ–ผ
                    โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
                    โ”‚         Question Classifier         โ”‚
                    โ”‚   (ๆŸฅ่ฉข้กžๅž‹ / ่ค‡้›œๅบฆ / ่กจๆ ผ็ฏ„ๅœๅˆคๆ–ท)    โ”‚
                    โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
                                      โ”‚
                โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
                โ–ผ                     โ–ผ                     โ–ผ
        โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”      โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”      โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
        โ”‚    Schema     โ”‚      โ”‚   Few-Shot   โ”‚      โ”‚   Business   โ”‚
        โ”‚   Retriever   โ”‚      โ”‚   Selector   โ”‚      โ”‚     Rules    โ”‚
        โ”‚  (vector DB)  โ”‚      โ”‚   (RAG)      โ”‚      โ”‚   Injector   โ”‚
        โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜      โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜      โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
                โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
                                      โ–ผ
                    โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
                    โ”‚         Prompt Composer             โ”‚
                    โ”‚   (system + schema + examples + Q)  โ”‚
                    โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
                                      โ”‚
                                      โ–ผ
                    โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
                    โ”‚         LLM Router                  โ”‚
                    โ”‚  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚
                    โ”‚  โ”‚  Azure   โ”‚   AWS    โ”‚   GCP    โ”‚ โ”‚
                    โ”‚  โ”‚  OpenAI  โ”‚ Bedrock  โ”‚  Vertex  โ”‚ โ”‚
                    โ”‚  โ”‚ (primary)โ”‚  (POC)   โ”‚  (POC)   โ”‚ โ”‚
                    โ”‚  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚
                    โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
                                      โ”‚
                                      โ–ผ
                    โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
                    โ”‚      SQL Validator & Sanitizer      โ”‚
                    โ”‚   (read-only, schema check, sandbox)โ”‚
                    โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
                                      โ”‚
                                      โ–ผ
                    โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
                    โ”‚       Database Executor             โ”‚
                    โ”‚   (MySQL / PostgreSQL / MSSQL)      โ”‚
                    โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
                                      โ”‚
                                      โ–ผ
                    โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
                    โ”‚      Result Formatter & Cache       โ”‚
                    โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜

ๆ ธๅฟƒๆŒ‘ๆˆฐไธ€:Schema ่™•็†

็›ดๆŽฅๆŠŠ schema ๅ…จๅกž็ตฆ LLM ๆ˜ฏๆ–ฐๆ‰‹ๆœ€ๅธธ็Šฏ็š„้Œฏ่ชคใ€‚ๆˆ‘ๅ€‘็š„่ณ‡ๆ–™ๅบซๆœ‰ 200+ ่กจ,full schema ๅคง็ด„ 60K token,ๆฏๆฌกๆŸฅ่ฉข้ƒฝ้€ไธ€ไปฝๆˆๆœฌๅฎŒๅ…จไธๅฏๆŽงใ€‚

ๆˆ‘ๅ€‘็š„ๅšๆณ•:Schema-as-Vector

  1. ๅปบ็ดขๅผ•ๆ™‚:ๆŠŠๆฏๅ€‹่กจ็ตๆง‹่ฝ‰ๆˆ่‡ช็„ถ่ชž่จ€ๆ่ฟฐ,็”จ embedding model ็ทจ็ขผๅˆฐ vector DB

    ่กจๅ:order_items
    ๆ่ฟฐ:่จ‚ๅ–ฎๆ˜Ž็ดฐ่กจ,่จ˜้Œ„ๆฏ็ญ†่จ‚ๅ–ฎๅŒ…ๅซ็š„ๅ•†ๅ“ใ€ๆ•ธ้‡ใ€ๅ–ฎๅƒน
    ไธป่ฆๆฌ„ไฝ:order_id (FK), product_id (FK), quantity, unit_price
    ้—œ่ฏ:JOIN orders ON order_items.order_id = orders.id
         JOIN products ON order_items.product_id = products.id
    
  2. ๆŸฅ่ฉขๆ™‚:ๆŠŠไฝฟ็”จ่€…ๅ•้กŒ็ทจ็ขผ,ๅพž vector DB ๅ–ๆœ€็›ธ้—œ็š„ 5-10 ๅ€‹่กจ

  3. ๅ‹•ๆ…‹็ต„่ฃ prompt:ๅชๆŠŠ้€™ไบ›่กจ็š„ schema ๆ”พ้€ฒ prompt,token ๅพž 60K ้™ๅˆฐ 3-5K

ๆ•ˆๆžœ:ๅนณๅ‡ token ็”จ้‡้™ไฝŽ 92%,ไธ” LLM ไธๆœƒ่ขซ็„ก้—œ schema ๅนฒๆ“พใ€‚


ๆ ธๅฟƒๆŒ‘ๆˆฐไบŒ:Prompt Engineering

Text2SQL ไธๆ˜ฏใ€ŒไธŸๅ€‹ๅ•้กŒ็ตฆ LLMใ€ๅฐฑๆœ‰็ญ”ๆกˆ,prompt ่จญ่จˆ็ดฐ็ฏ€็›ดๆŽฅๆฑบๅฎšๆบ–็ขบ็އใ€‚

Few-Shot Example ๅ‹•ๆ…‹้ธๆ“‡

ๆˆ‘ๅ€‘็ถญ่ญทไธ€ๅ€‹็ฏ„ไพ‹ๅบซ(็ด„ 200 ๅ€‹ question-SQL pair),ๆฏๆฌกๆŸฅ่ฉขๆ™‚:

  1. ่จˆ็ฎ—ไฝฟ็”จ่€…ๅ•้กŒ่ˆ‡ๆฏๅ€‹็ฏ„ไพ‹็š„ embedding ็›ธไผผๅบฆ
  2. ๅ– top-5 ๆœ€็›ธไผผ็š„็ฏ„ไพ‹ๆ”พ้€ฒ prompt
  3. ็ขบไฟ 5 ๅ€‹็ฏ„ไพ‹ๆถต่“‹ไธๅŒ็š„ SQL ๆจกๅผ(JOINใ€่šๅˆใ€ๅญๆŸฅ่ฉขใ€่ฆ–็ช—ๅ‡ฝๆ•ธ)

CoT (Chain-of-Thought) Prompt ็ตๆง‹

ไฝ ๆ˜ฏไธ€ไฝ่ณ‡ๆ–™ๅบซๅฐˆๅฎถใ€‚่ซ‹ๆ นๆ“šไปฅไธ‹ schema ่ˆ‡็ฏ„ไพ‹,ๅฐ‡ไฝฟ็”จ่€…ๅ•้กŒ่ฝ‰ๆ›็‚บ SQLใ€‚

[Schema ๅ€]
{retrieved_schemas}

[็ฏ„ไพ‹ๅ€]
{few_shot_examples}

[ๆฅญๅ‹™่ฆๅ‰‡ๅ€]
- ่จ‚ๅ–ฎ็‹€ๆ…‹ status=1 ่กจ็คบๅทฒไป˜ๆฌพ
- ๅ•†ๅ“่กจ deleted_at ไธ็‚บ NULL ่กจ็คบๅทฒไธ‹ๆžถ,ๆŸฅ่ฉขๆ™‚่ซ‹ๆŽ’้™ค
- ๆ—ฅๆœŸๆฌ„ไฝ็š†็‚บ UTC,้œ€่ฝ‰ๆ›็‚บ +08:00

[ไฝฟ็”จ่€…ๅ•้กŒ]
{user_question}

่ซ‹ไพไปฅไธ‹ๆญฅ้ฉŸๆ€่€ƒ:
1. ่ญ˜ๅˆฅๅ•้กŒๆถ‰ๅŠๅ“ชไบ›่กจ
2. ่ฆๅŠƒ JOIN ้ †ๅบ่ˆ‡ๆขไปถ
3. ๅฅ—็”จๅฟ…่ฆ็š„ WHERE ้Žๆฟพ(ๅŒ…ๅซๆฅญๅ‹™่ฆๅ‰‡)
4. ๆฑบๅฎš GROUP BYใ€ORDER BYใ€LIMIT
5. ่ผธๅ‡บๆœ€็ต‚ SQL(ๅชๅ›ž SQL,ไธ่ฆ่งฃ้‡‹)

CoT ่ฎ“ๆบ–็ขบ็އๆๅ‡็ด„ 15%,ไปฃๅƒนๆ˜ฏ token ็•ฅๅขžใ€‚


ๆ ธๅฟƒๆŒ‘ๆˆฐไธ‰:ๅคš้›ฒ LLM ๅˆ‡ๆ›็ญ–็•ฅ

ๅช็”จๅ–ฎไธ€ LLM ๆœ‰ไธ‰ๅคง้ขจ้šช:

  • ๅ–ฎ้ปžๆ•…้šœ(2024 ๅนด GPT-4 ๆ›พ้€ฃ็บŒ 4 ๅฐๆ™‚ outage)
  • ๆˆๆœฌ็ถๆญป(ไพ›ๆ‡‰ๅ•†ๆผฒๅƒนๅช่ƒฝๅž)
  • ่ƒฝๅŠ›ๅทฎ็•ฐ(ๆŸไบ› query ็”จ Claude ่กจ็พๆ›ดๅฅฝ)

Router ่จญ่จˆ

class LLMRouter:
    def route(self, question: str, complexity: str) -> LLMProvider:
        # ็ฐกๅ–ฎๆŸฅ่ฉข็”จไพฟๅฎœๆจกๅž‹
        if complexity == "simple":
            return AzureOpenAI("gpt-4o-mini")  # ไธปๅŠ›,ๆˆๆœฌไฝŽ

        # ่ค‡้›œๆŸฅ่ฉข็”จๅผทๆจกๅž‹
        if complexity == "complex":
            return AzureOpenAI("gpt-4o")  # ไธปๅŠ›

        # A/B test ๆต้‡ๅˆ‡ๅˆฐ POC ๅนณๅฐ
        if self.in_ab_test_group(question):
            return random.choice([
                AwsBedrock("claude-sonnet-4-5"),
                GcpVertex("gemini-2.0-pro")
            ])

        return AzureOpenAI("gpt-4o")  # ้ ่จญ

Fallback ๆฉŸๅˆถ

ไธป LLM ๅคฑๆ•—(timeout / rate limit / API ้Œฏ่ชค)ๆ™‚,่‡ชๅ‹•ๅˆ‡ๅˆฐๅ‚™ๆด:

Azure OpenAI โ†’ AWS Bedrock โ†’ GCP Vertex AI โ†’ ้™็ดšๅˆฐ cached response

ๆฏๆฌกๅˆ‡ๆ›้ƒฝ่จ˜้Œ„ๅˆฐ OpenTelemetry,ๆ–นไพฟ SRE ็›ฃๆŽงใ€‚

ๆณจๆ„:Bedrock ่ˆ‡ Vertex AI ๅœจๆˆ‘ๅ€‘็ณป็ตฑไธญ้‚„ๅœจ POC ้šŽๆฎต,production ไธปๅŠ›ไปๆ˜ฏ Azure OpenAIใ€‚ๅคš้›ฒๆ˜ฏใ€Œๅ‚™ๆด่ˆ‡ๆˆๆœฌๅ„ชๅŒ–ใ€็š„็ญ–็•ฅ,ไธๆ˜ฏใ€ŒๅŒๆ™‚ไธ‰ๅฎถๅ…จ่ท‘ใ€ใ€‚


ๆ ธๅฟƒๆŒ‘ๆˆฐๅ››:SQL ้ฉ—่ญ‰่ˆ‡ๅฎ‰ๅ…จ

LLM ็”Ÿๅ‡บ็š„ SQL ็ต•ๅฐไธ่ƒฝ็›ดๆŽฅๅŸท่กŒ,ๆœ‰ไธ‰ๅคง้ขจ้šช:

  1. SQL injection(LLM ๅฏ่ƒฝ่ขซไฝฟ็”จ่€…ๅ•้กŒๆณจๅ…ฅ)
  2. ไธๅฏ้€†ๆ“ไฝœ(LLM ๅฏซๅ‡บ DELETE / UPDATE / DROP)
  3. ๆ•ˆ่ƒฝๆฎบๆ‰‹(LLM ๅฏซๅ‡บ cross join ๆŠŠ DB ๆ‹–ๅžฎ)

้ฉ—่ญ‰ๆตๆฐด็ทš

LLM SQL Output
     โ”‚
     โ–ผ
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚ 1. AST ่งฃๆž (sqlparse)      โ”‚
โ”‚    - ๆ˜ฏๅฆ็‚บ SELECT?          โ”‚
โ”‚    - ๆ˜ฏๅฆๅซ DDL/DML?         โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
          โ”‚ pass
          โ–ผ
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚ 2. Schema ้ฉ—่ญ‰              โ”‚
โ”‚    - ่กจๅ/ๆฌ„ไฝๆ˜ฏๅฆๅญ˜ๅœจ?       โ”‚
โ”‚    - JOIN ๆขไปถๆ˜ฏๅฆๅˆๆณ•?       โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
          โ”‚ pass
          โ–ผ
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚ 3. EXPLAIN PLAN ๅˆ†ๆž        โ”‚
โ”‚    - ไผฐ่จˆ row count          โ”‚
โ”‚    - ๆ˜ฏๅฆ่ตฐ full scan?       โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
          โ”‚ pass
          โ–ผ
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚ 4. Read-only ๅธณ่™ŸๅŸท่กŒ        โ”‚
โ”‚    - ่จญๅฎš query timeout      โ”‚
โ”‚    - ้™ๅˆถ result row limit   โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜

ไปปไธ€ๆญฅๅคฑๆ•—ๅฐฑ reject ไธฆ้‡ๆ–ฐ็”Ÿๆˆ SQLใ€‚


Evaluation:็ง‘ๅญธๅœฐๆธฌๆบ–็ขบ็އ

ๅพˆๅคšไบบๅšๅฎŒ Text2SQL ๅฐฑ่ชชใ€Œๆˆ‘่ฆบๅพ—่ ปๆบ–็š„ใ€,ไฝ†ใ€Œ่ฆบๅพ—ใ€ไธๆ˜ฏๅทฅ็จ‹ใ€‚

Golden Set ๅปบ็ซ‹

ๆˆ‘ๅ€‘่Šฑ 2 ้€ฑๅปบ็ซ‹ 150 ้กŒ Golden Set:

  • ๆถต่“‹ simple / medium / complex ไธ‰็จฎ้›ฃๅบฆ
  • ๆฏ้กŒ็”ฑๆฅญๅ‹™ๅŒไบ‹ๆไพ›ๅ•้กŒ,DBA ๅฏซๆจ™ๆบ– SQL
  • ๆถต่“‹ JOINใ€่šๅˆใ€ๅญๆŸฅ่ฉขใ€่ฆ–็ช—ๅ‡ฝๆ•ธใ€ๆ—ฅๆœŸๅ‡ฝๆ•ธๅ…ญๅคง้กž

่ฉ•ไผฐๆŒ‡ๆจ™

ๆŒ‡ๆจ™ ๅฎš็พฉ
Execution Accuracy LLM ็”Ÿๆˆ็š„ SQL ่ท‘ๅ‡บไพ†็š„็ตๆžœๆ˜ฏๅฆ่ˆ‡ Golden SQL ็ตๆžœไธ€่‡ด
Exact Match LLM ็”Ÿๆˆ็š„ SQL ๆ˜ฏๅฆ่ˆ‡ Golden SQL ๅญ—ไธฒๅฎŒๅ…จ็›ธๅŒ(ๅšดๆ ผ)
Semantic Match LLM ็”Ÿๆˆ็š„ SQL ็ตๆง‹ๆ˜ฏๅฆ่ˆ‡ Golden SQL ็ญ‰ๅƒน(ๅ…่จฑ alias ไธๅŒ)

CI ๆ•ดๅˆ

ๆฏๆฌกๆ”น prompt ๆˆ–ๆ›ๆจกๅž‹,่‡ชๅ‹•่ท‘ Golden Set ไธฆๆฏ”ๅฐ:

pytest tests/eval/test_text2sql.py --eval-set golden_v3
# Output:
# Simple:  98.5% (148/150)
# Medium:  91.2%
# Complex: 76.8%
# Total:   88.9% Execution Accuracy

ไฝŽๆ–ผ baseline ๅฐฑ block PR,็ขบไฟๆ”นๅ‹•ไธๆœƒ regressionใ€‚


่ธฉ้Ž็š„ๅ‘

ๅ‘ 1:LLM ๅนป่ฆบๅ‡บไธๅญ˜ๅœจ็š„ๆฌ„ไฝ

็—‡็‹€:LLM ใ€Œ่…ฆ่ฃœใ€ไบ†ไธ€ๅ€‹ customer_email ๆฌ„ไฝ,ไฝ†ๅฏฆ้š›่กจๅชๆœ‰ email ๅฐ็ญ–:Schema ้ฉ—่ญ‰้šŽๆฎต็›ดๆŽฅ reject,ไธฆๆŠŠ้Œฏ่ชคๆฌ„ไฝ + ๆญฃ็ขบๆฌ„ไฝๅˆ—่กจ้คตๅ›žๅŽป้‡็”Ÿ

ๅ‘ 2:ๅคง schema ็š„ token ็ˆ†็‚ธ

็—‡็‹€:200+ ่กจ็š„ schema ๅ…จๅกž,ๆฏๆฌกๆŸฅ่ฉข 60K token,ๆˆๆœฌไธ€ๅ€‹ๆœˆ็‡’ๆމๅนพ่ฌ็พŽๅ…ƒ ๅฐ็ญ–:Schema-as-Vector(ไธŠ้ข่ฌ›้Ž)

ๅ‘ 3:Prompt ็‰ˆๆœฌ็ฎก็†ๅคฑๆŽง

็—‡็‹€:ไธ‰ๅ€‹ๅทฅ็จ‹ๅธซๅŒๆ™‚ๆ”น prompt,ๆฒ’ไบบ็Ÿฅ้“ๅ“ช็‰ˆ็š„ๆบ–็ขบ็އๆœ€้ซ˜ ๅฐ็ญ–:ๆŠŠ prompt ็•ถๆˆ code ็ฎก็†,ๅฏซๅ…ฅ Git,ๆฏ็‰ˆ่ท‘ Golden Set,็ตๆžœๅญ˜ๅˆฐ metric DB

ๅ‘ 4:ๅคš่ชž่จ€ๆทท้›œ

็—‡็‹€:ไฝฟ็”จ่€…ๅ•ใ€Œ2024 ๅนด็ฌฌไธ€ๅญฃ็š„ revenueใ€,LLM ๆŠŠใ€Œrevenueใ€็•ถ่‹ฑๆ–‡ไธ็ฟป่ญฏ,ไฝ† SQL ๆฌ„ไฝๆ˜ฏ ็ธฝ็‡Ÿๆ”ถ ๅฐ็ญ–:Schema ๆ่ฟฐๅŒๆ™‚็”จไธญ่‹ฑ้›™่ชžๆจ™่จ˜,ไธฆๅœจ system prompt ๆ้†’ใ€Œ่ก“่ชžๅฐ็…ง่กจใ€


็ต่ชž:AI-Driven Development ็š„ๆ–ฐ้ซ”ๆ„Ÿ

้€™ๅ€‹ๅฐˆๆกˆ็š„ๅฆไธ€ๅฑคๆ”ถ็ฉซ,ๆ˜ฏๅ…จ็จ‹ไฝฟ็”จ Cursor ่ˆ‡ Claude Code ้–‹็™ผ็š„้ซ”้ฉ—ใ€‚

้ŽๅŽปๅฏซไธ€ๅ€‹่ค‡้›œ็š„ retriever ๆจก็ต„,ๅฏ่ƒฝ่ฆ่Šฑ 2-3 ๅคฉ็œ‹ LangChain ๆ–‡ไปถใ€ๅฏซๆธฌ่ฉฆใ€debugใ€‚็พๅœจ็”จ Cursor ็š„ multi-file edit + Claude ็š„ agentic mode,ๅพž่จญ่จˆๅˆฐๆธฌ่ฉฆๅคงๆฆ‚ 4-6 ๅฐๆ™‚ๅฐฑ่ƒฝๅฎŒๆˆไธ€ๅ€‹ๅฏ็”จ็š„็‰ˆๆœฌใ€‚

ไฝ†้€™ไธไปฃ่กจ AI ๅ–ไปฃๅทฅ็จ‹ๅธซใ€‚ๆˆ‘็™ผ็พ็š„้—œ้ตๆ˜ฏ:

AI ๅทฅๅ…ทๆ”พๅคง็š„ๆ˜ฏๅทฅ็จ‹ๅธซ็š„ใ€Œๅ“ๅ‘ณใ€โ€”โ€”ไฝ ่ถŠ็Ÿฅ้“ๅฅฝ็จ‹ๅผ้•ทไป€้บผๆจฃ,AI ็ตฆไฝ ็š„ๅƒนๅ€ผ่ถŠ้ซ˜;่ถŠไธๆœƒ review ็จ‹ๅผ็ขผ,AI ็ตฆไฝ ็š„ๅฐฑๅชๆ˜ฏไธ€ๅจ OK-ish ็š„ spaghettiใ€‚

ๅฆ‚ๆžœๆœ‰ไป€้บผไธ€ๅฅ่ฉฑๅฟƒๅพ—ๅธถ่ตฐ,ๆˆ‘ๆœƒ่ชช:ๅฅฝ็š„ๅทฅ็จ‹ๅธซไธๅช็”จ AI,้‚„่ฆ review AIใ€‚


ๅพŒ็บŒๆ–นๅ‘

  • ๅŠ ๅ…ฅ Schema ่‡ชๅ‹•ๆ›ดๆ–ฐๆฉŸๅˆถ(่ณ‡ๆ–™ๅบซ่ฎŠๆ›ดๆ™‚่‡ชๅ‹•้‡ๅปบ vector index)
  • ๆŽข็ดข Multi-Agent ๆจกๅผ(่ฆๅŠƒ Agent + ็”Ÿๆˆ Agent + ้ฉ—่ญ‰ Agent ๅˆ†ๅทฅ)
  • ๆ•ดๅˆ MCP Protocol ่ฎ“ Text2SQL ่ฎŠๆˆๅฏๆ’ๆ‹”็š„ LLM tool
  • ๅปบ็ซ‹ Prompt A/B test ๅนณๅฐ,็ณป็ตฑๅŒ–่ฟญไปฃ

ไธŠ GitHub ๆ€Ž้บผๅš?

  1. ๅœจไฝ ็š„ GitHub ๅปบไธ€ๅ€‹ repo,ไพ‹ๅฆ‚ text2sql-engineering-notes
  2. ๆŠŠ้€™็ฏ‡ .md ๅ‘ฝๅ็‚บ README.md ๆˆ– text2sql-from-zero.md
  3. ๅŠ ไธŠๆžถๆง‹ๅœ–(ๅฏไปฅ็”จ mermaid ้‡็•ซ,ๆˆ–ไธŠๅ‚ณ PNG)
  4. Push ไธŠๅŽป
  5. ้‡้ปž:ๅœจไฝ ็š„ GitHub Profile README ่ฃก pin ้€™ๅ€‹ repo

ๆˆ–่€…ๆ›ด้€ฒ้šŽ็š„ๅšๆณ•:็”จ GitHub Pages ๅš blog

  • Repo ๅปบ your-username.github.io
  • ็”จ Jekyllใ€Hugoใ€Astro ไปปไธ€ๆก†ๆžถ
  • ๆŠŠ้€™็ฏ‡ๆ”พ้€ฒ _posts/ ๆˆ– content/posts/
  • ่‡ชๅ‹•็”Ÿๆˆ blog ็ถฒ็ซ™

ๆœ‰่ˆˆ่ถฃ่จŽ่ซ–็š„่ฉฑ?

About

Multi-LLM Text2SQL engine for cross-database natural language querying (MySQL/PostgreSQL/MSSQL). Schema-as-Vector retrieval reduces token usage by 92%. Azure OpenAI primary with AWS Bedrock & GCP Vertex AI POC.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors