rjmalagon / gte-qwen2-7b-instruct-embed-f16

4,203 撤回更新于8周前

更新于8周前

8周前

a94ce5b37c1c · 15GB

{{ if .System }}<|im_start|>系统 {{ .System }}<|im_end|> {{ end }}{{ if .Prompt }}<|im_start|>用户 {{ .Prompt }}<|im_end|> {{ end }}<|im_start|>助手 {{ .Response }}<|im_end|>

181B

README

gte-Qwen2-7B-instruct是gte（通用文本嵌入）模型系列中最新模型，在大规模文本嵌入基准MTEB（截至2024年6月16日）的英语和中文评估中均排名第一。

最近，Qwen 团队发布了 Qwen2 系列模型，我们基于 Qwen2-7B LLM 模型训练了 gte-Qwen2-7B-instruct 模型。与 gte-Qwen1.5-7B-instruct 模型相比，gte-Qwen2-7B-instruct 模型在微调阶段使用了相同的训练数据和训练策略，唯一的区别是将基础模型升级为 Qwen2-7B。考虑到 Qwen2 系列模型相较于 Qwen1.5 系列的改进，我们也可以期待嵌入模型性能的持续提升。

模型集成了几个关键的新进展

Integration of bidirectional attention mechanisms, enriching its contextual understanding.
Instruction tuning, applied solely on the query side for streamlined efficiency
Comprehensive training across a vast, multilingual text corpus spanning diverse domains and scenarios. This training leverages both weakly supervised and supervised data, ensuring the model's applicability across numerous languages and a wide array of downstream tasks.

模型信息

Model Size: 7B
Embedding Dimension: 3584
Max Input Tokens: 32k

https://hugging-face.cn/Alibaba-NLP/gte-Qwen2-7B-instruct

gte-Qwen2-7B-instruct is the latest model in the gte (General Text Embedding) model family that ranks No.1 in both English and Chinese evaluations on the Massive Text Embedding Benchmark MTEB benchmark (as of June 16, 2024).

Recently, the Qwen team released the Qwen2 series models, and we have trained the gte-Qwen2-7B-instruct model based on the Qwen2-7B LLM model. Compared to the gte-Qwen1.5-7B-instruct model, the gte-Qwen2-7B-instruct model uses the same training data and training strategies during the finetuning stage, with the only difference being the upgraded base model to Qwen2-7B. Considering the improvements in the Qwen2 series models compared to the Qwen1.5 series, we can also expect consistent performance enhancements in the embedding models.

The model incorporates several key advancements:

Integration of bidirectional attention mechanisms, enriching its contextual understanding.
    Instruction tuning, applied solely on the query side for streamlined efficiency
    Comprehensive training across a vast, multilingual text corpus spanning diverse domains and scenarios. This training leverages both weakly supervised and supervised data, ensuring the model's applicability across numerous languages and a wide array of downstream tasks.

Model Information

Model Size: 7B
    Embedding Dimension: 3584
    Max Input Tokens: 32k

[https://hugging-face.cn/Alibaba-NLP/gte-Qwen2-7B-instruct](https://hugging-face.cn/Alibaba-NLP/gte-Qwen2-7B-instruct)

Paste, drop or click to upload images (.png, .jpeg, .jpg, .svg, .gif)