Update README.md
Browse files
README.md
CHANGED
|
@@ -1,4 +1,3 @@
|
|
| 1 |
-
---
|
| 2 |
pipeline_tag: text-generation
|
| 3 |
license: apache-2.0
|
| 4 |
tags:
|
|
@@ -83,24 +82,14 @@ Search-based technology, AutoNAC.
|
|
| 83 |
|
| 84 |
## Model Architecture
|
| 85 |
|
| 86 |
-
| Parameters | Layers | Heads | Sequence Length | GQA num_key_value_heads |
|
| 87 |
-
|:----------|:----------|:----------|:----------|:----------|
|
| 88 |
-
| 6B | 32 | 32 | 4096 | Variable |
|
| 89 |
|
| 90 |
|
| 91 |
- **Decoder layer:** Variable Grouped Query Attention
|
| 92 |
- **Position Embeddings:** Rotary Position Embeddings [Su et al., 2021](https://arxiv.org/abs/2104.09864)
|
| 93 |
|
| 94 |
-
## Uses
|
| 95 |
-
|
| 96 |
-
The model is intended to perform single/multiline code completion from a
|
| 97 |
-
context window of up to 4096k tokens. It is *not* an instruction model
|
| 98 |
-
and commands like \"Write a function that computes the absolute value of
|
| 99 |
-
an integer,\" won't yield the desired results. A more effective approach
|
| 100 |
-
is to frame instructions in the style of source code comments (e.g. \#
|
| 101 |
-
this function calculates the absolute value of an integer) or to present
|
| 102 |
-
a function signature and docstring, enabling the model to complete the
|
| 103 |
-
function's body.
|
| 104 |
|
| 105 |
### How to Use
|
| 106 |
|
|
|
|
|
|
|
| 1 |
pipeline_tag: text-generation
|
| 2 |
license: apache-2.0
|
| 3 |
tags:
|
|
|
|
| 82 |
|
| 83 |
## Model Architecture
|
| 84 |
|
| 85 |
+
| Parameters | Layers | Heads | Sequence Length | GQA num_key_value_heads |
|
| 86 |
+
|:----------|:----------|:----------|:----------|:----------|
|
| 87 |
+
| 6B | 32 | 32 | 4096 | Variable |
|
| 88 |
|
| 89 |
|
| 90 |
- **Decoder layer:** Variable Grouped Query Attention
|
| 91 |
- **Position Embeddings:** Rotary Position Embeddings [Su et al., 2021](https://arxiv.org/abs/2104.09864)
|
| 92 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 93 |
|
| 94 |
### How to Use
|
| 95 |
|