taozhiyuai / openbiollm-llama-3

推动医疗领域开源大型语言模型的进步

8B 70B

853 拉取更新于 2 个月前

更新于 3 个月前

3 个月前

2a2f1787ceac · 58GB

{{ if .System }}<|start_header_id|>system<|end_header_id|> {{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|> {{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|> {{ .Response }}<|eot_id|>

{"num_keep":24,"stop":["<|start_header_id|>","<|end_header_id|>","<|eot_id|>"]}

按Markdown格式编排回复。

README

从https://hf-mirror.com/aaditya/Llama3-OpenBioLLM-70B导入

推动医疗领域开源大型语言模型的进步

对于多语言支持，访问https://ollama.ac.cn/taozhiyuai/openbiollm-llama-3-chinese

简介介绍

生物医学领域的佼佼者，基于LLAMA3打造。

引见 OpenBioLLM-70B：一款顶尖的开源生物医学大型语言模型

OpenBioLLM-70B是一款专为生物医学领域设计的先进开源语言模型。由Saama AI实验室开发，该模型采用最尖端的技巧，在广泛的生物医学任务上实现了最先进的表现。

🏥 生物医学专业化：OpenBioLLM-70B针对医疗和生命科学领域的独特语言和知识需求量身定制。它在大规模高质量生物医学数据集上进行微调，能够以特定领域的准确性和流畅性理解和生成文本。

🎓 优异性能：拥有70亿参数，OpenBioLLM-70B在类似规模的其他开源生物医学语言模型中表现出色。它还在生物医学基准测试中，与GPT-4、Gemini、Meditron-70B、Med-PaLM-1和Med-PaLM-2等更大的私有和开源模型相比，取得了更好的结果。

🧠 先进训练技术：OpenBioLLM-70B建立在Meta-Llama-3-70B-Instruct和Meta-Llama-3-70B-Instruct模型强大的基础上。它采用了DPO数据集和微调配方，以及自定义多样的医疗指令数据集。训练流程的关键组件包括

示例屏幕样本

70B的输出内容质量高；

70B生成更多和高质量的文本

8B 的输出内容较少，且输出的中文质量不稳定，过度量化导致损失率高。建议使用70B。

8B 生成内容较少，量化越多，困惑度越高。因此推荐使用 70B。

| Model                        | Quants | Size  | Bit | Perplexity       |
|------------------------------|--------|-------|----|-------------------|
| llama3-openbiollm-8b:Q4_0   | Q4_0   | 4.7GB | 4  | +0.2166 ppl       |
| llama3-openbiollm-8b:Q4_K_M  | Q4_K_M | 4.9GB | 4  | +0.0532 ppl       |
| llama3-openbiollm-8b:Q5_K_M  | Q5_K_M | 5.7GB | 5  | +0.0122 ppl       |
| llama3-openbiollm-8b:Q6_K    | Q6_K   | 6.6GB | 6  | +0.0008 ppl       |

基准医疗模型测评

微信ID: TAOZHIYUAI

import from https://hf-mirror.com/aaditya/Llama3-OpenBioLLM-70B

![fJIOPJnY6Ff6fUiSIuMEt.png](https://ollama.ac.cn/assets/taozhiyuai/openbiollm-llama-3/7fe6ae7a-69b1-48a5-b4d8-8f19c214b209)
# **Advancing Open-source Large Language Models in Medical Domain**

## For mutil-language support, visit [https://ollama.ac.cn/taozhiyuai/openbiollm-llama-3-chinese](https://ollama.ac.cn/taozhiyuai/openbiollm-llama-3-chinese)

![KGmRE5w2sepNtwsEu8t7K.jpeg](https://ollama.ac.cn/assets/taozhiyuai/openbiollm-llama-3-chinese/e0571a63-85d2-400e-bde1-956b73ae7795)

# Introduction 介绍

生物医学领域优等生,基于LLAMA3打造.

Introducing OpenBioLLM-70B: A State-of-the-Art Open Source Biomedical Large Language Model

OpenBioLLM-70B is an advanced open source language model designed specifically for the biomedical domain. Developed by Saama AI Labs, this model leverages cutting-edge techniques to achieve state-of-the-art performance on a wide range of biomedical tasks.

🏥 Biomedical Specialization: OpenBioLLM-70B is tailored for the unique language and knowledge requirements of the medical and life sciences fields. It was fine-tuned on a vast corpus of high-quality biomedical data, enabling it to understand and generate text with domain-specific accuracy and fluency.

🎓 Superior Performance: With 70 billion parameters, OpenBioLLM-70B outperforms other open source biomedical language models of similar scale. It has also demonstrated better results compared to larger proprietary & open-source models like GPT-4, Gemini, Meditron-70B, Med-PaLM-1 & Med-PaLM-2 on biomedical benchmarks.

🧠 Advanced Training Techniques: OpenBioLLM-70B builds upon the powerful foundations of the Meta-Llama-3-70B-Instruct and Meta-Llama-3-70B-Instruct models. It incorporates the DPO dataset and fine-tuning recipe along with a custom diverse medical instruction dataset. Key components of the training pipeline include:

# Screen sample 示例

**70B**的输出内容多质量高 ;

**70B** generate more and quality texts

![截屏2024-05-08 22.07.00.png](https://ollama.ac.cn/assets/taozhiyuai/openbiollm-llama-3-chinese/2a2748a1-913d-47c3-a42e-71ea8c8b6bae)

**8B**的输出内容少,且输出的中文质量不稳定,过度量化导致损失率高. 建议使用70B.;

**8B** generate less, More quants more perplexity. so **70B** is recommended.

```markdown
| Model                        | Quants | Size  | Bit | Perplexity       |
|------------------------------|--------|-------|----|-------------------|
| llama3-openbiollm-8b:Q4_0   | Q4_0   | 4.7GB | 4  | +0.2166 ppl       |
| llama3-openbiollm-8b:Q4_K_M  | Q4_K_M | 4.9GB | 4  | +0.0532 ppl       |
| llama3-openbiollm-8b:Q5_K_M  | Q5_K_M | 5.7GB | 5  | +0.0122 ppl       |
| llama3-openbiollm-8b:Q6_K    | Q6_K   | 6.6GB | 6  | +0.0008 ppl       |
```

![截屏2024-05-09 15.21.49.png](https://ollama.ac.cn/assets/taozhiyuai/openbiollm-llama-3-chinese/534942de-d746-41c3-9152-c51161851b74)

#  Benchmark 医疗模型测评

![oPchsJsEpQoGcGXVbh7YS.png](https://ollama.ac.cn/assets/taozhiyuai/openbiollm-llama-3-chinese/9ae6aee7-8d6e-456d-a127-fc33753b2c4f)
![UXF-V0col0Z0sS6BGPBkE.png](https://ollama.ac.cn/assets/taozhiyuai/openbiollm-llama-3-chinese/4d3ec1a6-03a8-4f32-948f-0166a61eedc9)
![截屏2024-05-09 14.25.03.png](https://ollama.ac.cn/assets/taozhiyuai/openbiollm-llama-3-chinese/feabad56-cd7b-48ba-9297-353c9fe2691d)

# **WeChat ID: TAOZHIYUAI**

Paste, drop or click to upload images (.png, .jpeg, .jpg, .svg, .gif)