Add Llama.cpp support!

[methedude]

This harness does not work with llama.cpp!

[rehan243]

Yeah, llama.cpp integration’s tricky since it doesn’t follow the usual Hugging Face API most of this harness assumes. We ran into something similar with a custom GPTQ backend—ended up wrapping it with a lightweight adapter to mimic the expected interface. Honestly, you’re probably looking at doing the same thing unless they decide to refactor for native llama.cpp support.

[FazeelUsmani]

Based on the code, the hang likely occurs during post-generation phases like lm.barrier(), lm.gather_object(), or _process_results(). If you're using multiple GPUs, this could be a distributed deadlock.