Skip to content

ds4_agent: fix rocm support#383

Open
mcmalayalam wants to merge 1 commit into
antirez:mainfrom
mcmalayalam:ds4-agent_fix_rocm
Open

ds4_agent: fix rocm support#383
mcmalayalam wants to merge 1 commit into
antirez:mainfrom
mcmalayalam:ds4-agent_fix_rocm

Conversation

@mcmalayalam

Copy link
Copy Markdown

Fixes ds4-agent so that it can run with --backend rocm.

Testing evidence

Successfully ran ds4-agent on a Framework Desktop:

$ ./ds4-agent \
        --backend rocm \
        --threads 30 \
        --model ~/models/antirez/deepseek-v4-gguf/DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix.gguf \
        --mtp ~/models/antirez/deepseek-v4-gguf/DeepSeek-V4-Flash-MTP-Q4K-Q8_0-F32.gguf \
        --mtp-draft 1 \
        --ctx 16000 \
ds4: MTP support model loaded: /home/sam/models/antirez/deepseek-v4-gguf/DeepSeek-V4-Flash-MTP-Q4K-Q8_0-F32.gguf (draft=1)
ds4: ROCm backend initialized on AMD Radeon Graphics (sm_115)
ds4: ROCm chunk-copying 80.76 GiB model image
ds4: ROCm loading model tensors into device cache: 80.06 GiBds4: ROCm model chunk copy complete in 18.271s (80.76 GiB tensors)
ds4: ROCm chunk-copying 3.55 GiB model image
ds4: ROCm loading model tensors into device cache: 2.06 GiBds4: ROCm model chunk copy complete in 0.782s (3.55 GiB tensors)
ds4: ROCm preparing model tensor mappings: 80.24 GiB
ds4: ROCm startup model preparation covered 80.76 GiB of tensor spans in 0.694s
ds4: ROCm preparing model tensor mappings: 2.38 GiB
ds4: ROCm startup model preparation covered 3.55 GiB of tensor spans in 0.012s
ds4: rocm backend initialized for graph diagnostics
DwarfStar 🐋 Agent, context 16k tokens


ds4-agent> 

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant