rjmalagon / gte-qwen2-7b-instruct-embed-f16

4,203 引用更新于8周前

更新于8周前

8周前

a94ce5b37c1c · 15GB

{{ if .System }}<|im_start|>system {{ .System }}<|im_end|> {{ end }}{{ if .Prompt }}<|im_start|>user {{ .Prompt }}<|im_end|> {{ end }}<|im_start|>assistant {{ .Response }}<|im_end|>

181B

README

gte-Qwen2-7B-instruct是gte（通用文本嵌入）模型系列中最新的模型，在大量文本嵌入基准MTEB上（截至2024年6月16日）在英语和中文评估中均排名第一。

最近，Qwen团队发布了Qwen2系列模型，我们基于Qwen2-7B LLM模型训练了gte-Qwen2-7B-instruct模型。与gte-Qwen1.5-7B-instruct模型相比，gte-Qwen2-7B-instruct模型在微调阶段的训练数据和策略相同，唯一不同的是升级了基础模型到Qwen2-7B。考虑到Qwen2系列模型相对于Qwen1.5系列的改进，我们也可以期待嵌入模型的一致性能提升。

该模型融合了几个关键进步

Integration of bidirectional attention mechanisms, enriching its contextual understanding.
Instruction tuning, applied solely on the query side for streamlined efficiency
Comprehensive training across a vast, multilingual text corpus spanning diverse domains and scenarios. This training leverages both weakly supervised and supervised data, ensuring the model's applicability across numerous languages and a wide array of downstream tasks.

模型信息

Model Size: 7B
Embedding Dimension: 3584
Max Input Tokens: 32k

https://hugging-face.cn/Alibaba-NLP/gte-Qwen2-7B-instruct

gte-Qwen2-7B-instruct is the latest model in the gte (General Text Embedding) model family that ranks No.1 in both English and Chinese evaluations on the Massive Text Embedding Benchmark MTEB benchmark (as of June 16, 2024).

Recently, the Qwen team released the Qwen2 series models, and we have trained the gte-Qwen2-7B-instruct model based on the Qwen2-7B LLM model. Compared to the gte-Qwen1.5-7B-instruct model, the gte-Qwen2-7B-instruct model uses the same training data and training strategies during the finetuning stage, with the only difference being the upgraded base model to Qwen2-7B. Considering the improvements in the Qwen2 series models compared to the Qwen1.5 series, we can also expect consistent performance enhancements in the embedding models.

The model incorporates several key advancements:

Integration of bidirectional attention mechanisms, enriching its contextual understanding.
    Instruction tuning, applied solely on the query side for streamlined efficiency
    Comprehensive training across a vast, multilingual text corpus spanning diverse domains and scenarios. This training leverages both weakly supervised and supervised data, ensuring the model's applicability across numerous languages and a wide array of downstream tasks.

Model Information

Model Size: 7B
    Embedding Dimension: 3584
    Max Input Tokens: 32k

[https://hugging-face.cn/Alibaba-NLP/gte-Qwen2-7B-instruct](https://hugging-face.cn/Alibaba-NLP/gte-Qwen2-7B-instruct)

Paste, drop or click to upload images (.png, .jpeg, .jpg, .svg, .gif)