HQQ 3-bit w integrated LoRA APIs - BC official HQQ integration isn't here yet - RN investigating if 2bit is wrong or just always bad
👋 Open to Work
crumb
crumb
AI & ML interests
None yet
Recent Activity
updated a model about 2 hours ago
crumbs-playground/test-bf-moe updated a dataset about 17 hours ago
crumb/bf_random_valid_dr4l2_swarm_pre liked a model 3 days ago
XingChen-AGI/Xing4.0-29B-A4BOrganizations
First Prototype of the second iteration of MoLora utilizing mixture of expert techniques applied to the Llama2 model.
Parts of Meta's LlamaV2 models, chopped up and trained. CoreX means the first X layers were kept.
Allowing for the LM to learn a soft-"multi-step program" to predict future tokens instead of learning to predict future tokens itself.
HQQ 3-bit w integrated LoRA APIs - BC official HQQ integration isn't here yet - RN investigating if 2bit is wrong or just always bad
Model assets for the first Mixture-of-Lora technique applied to Llama. https://bit.ly/48bqshl
First Prototype of the second iteration of MoLora utilizing mixture of expert techniques applied to the Llama2 model.
GPT2 Models using Linear layers instead of Conv layers for convenience.
Parts of Meta's LlamaV2 models, chopped up and trained. CoreX means the first X layers were kept.
Smaller models trained locally on my 2xA6000 Lambda Vector
Allowing for the LM to learn a soft-"multi-step program" to predict future tokens instead of learning to predict future tokens itself.