This is a 2-bit quantized Llama2-7B model using FrameQuant. Use the inference script at https://github.com/vsingh-group/FrameQuant to run the model.
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support