Skip to content

Fix missing tree_reduce import in models/cache.py#1165

Merged
angeloskath merged 1 commit into
ml-explore:mainfrom
siiea-ai:fix/cache-import-tree-reduce
Apr 20, 2026
Merged

Fix missing tree_reduce import in models/cache.py#1165
angeloskath merged 1 commit into
ml-explore:mainfrom
siiea-ai:fix/cache-import-tree-reduce

Conversation

@siiea-ai
Copy link
Copy Markdown
Contributor

_BaseCache.nbytes calls tree_reduce, but the module only imports tree_flatten, tree_map, tree_unflatten from mlx.utils. Reading .nbytes on any cache that inherits the base implementation (hit by MoE architectures like Qwen3.5 MoE) raises NameError: name 'tree_reduce' is not defined.

Fixes #1164.

Diff

-from mlx.utils import tree_flatten, tree_map, tree_unflatten
+from mlx.utils import tree_flatten, tree_map, tree_reduce, tree_unflatten

Repro (before the fix)

curl -sS -X POST http://127.0.0.1:8888/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"qwen3.5-397b-a17b-mlx","stream":false,"max_tokens":20,
       "messages":[{"role":"user","content":"Say hello."}]}'
# → {"error":"Error in iterating prediction stream: NameError: name 'tree_reduce' is not defined"}

After the patch, the same request returns a normal completion. Verified on LM Studio 0.4.12+1 with bundled mlx-lm 0.31.3 on macOS Apple Silicon.

_BaseCache.nbytes calls tree_reduce but the module only imported
tree_flatten, tree_map, tree_unflatten from mlx.utils. Reading
.nbytes on any cache that inherits the base implementation
(hit by MoE architectures like Qwen3.5 MoE) raised
NameError: name 'tree_reduce' is not defined.

Fixes ml-explore#1164.
Copy link
Copy Markdown
Member

@angeloskath angeloskath left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@angeloskath angeloskath merged commit a401730 into ml-explore:main Apr 20, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

NameError: tree_reduce is not defined in mlx_lm/models/cache.py _BaseCache.nbytes

2 participants