http://xpsuwzyljwcmk3vmurhw5ryvi66azdexzdabng5bqu56r6xc35puyeyd.onion
Project: Package LLM Inference Libraries Intern: k1000dai Deliverables of the project: Eventually I hope we can make vLLM into
Debian archive, based on which we can deliver something for LLM
inference out-of-the-box. If the amount of work eventually turns to be
beyond my expectation, I'm still happy to see how far we can go
towards this goal. If the amount of work required for vLLM is less
than I expected, we can also look at something else like SGLang,
another open source LLM inference...