Issues / #72
Distributed model loading across forest machines using Soil shared VRAM
proposed
feature
Priority: medium
Project: nimsforest2
Reporter: anonymous
24 Mar 2026 14:19
Description
Investigate using Soil (or another cross-machine KV store) to load AI models across multiple machines in the forest, pooling their VRAM for larger model support.
Reference implementation: https://github.com/pryzmatpl/prismalama — a distributed LLM loading approach that could inform the design.
Reference implementation: https://github.com/pryzmatpl/prismalama — a distributed LLM loading approach that could inform the design.
Comments (1)
nebula
24 Mar 2026 14:30
Grooming: recategorized from bug to feature, set priority to medium, changed project from cindy to nimsforest2, rewrote truncated title, cleaned up description
Nebula's reasoning: This is a feature suggestion about distributed model loading using the Soil layer to pool VRAM across forest machines — not a bug. Recategorized to feature. The project belongs under nimsforest2 since it involves the Soil data layer (cross-machine KV), not cindy. Priority set to medium: this is a compelling architectural idea for scaling model inference, but it's exploratory and not blocking current workflows. The original title was the first sentence of the description truncated with "...", replaced with a clear summary. Description restructured from conversational note into an actionable investigation item while preserving the reference link.
Nebula's reasoning: This is a feature suggestion about distributed model loading using the Soil layer to pool VRAM across forest machines — not a bug. Recategorized to feature. The project belongs under nimsforest2 since it involves the Soil data layer (cross-machine KV), not cindy. Priority set to medium: this is a compelling architectural idea for scaling model inference, but it's exploratory and not blocking current workflows. The original title was the first sentence of the description truncated with "...", replaced with a clear summary. Description restructured from conversational note into an actionable investigation item while preserving the reference link.