pxlksr / defog_sqlcoder-7b-2

一种自然语言到SQL生成的强大大型语言模型。

1,043 Pulls 更新于5个月前

更新于5个月前

5个月前

7e94a4187864 · 13GB

README

许可协议：cc-by-sa-4.0

更新公告

该模型权重于2024年2月7日早上7点UTC更新。新的模型权重带来性能显著提升 - 特别是在连接操作方面。

如果您在此之前下载了该模型，请重新下载权重以获得最佳性能。

SQLCoder-7B-2 模型卡片

一种自然语言到SQL生成的强大大型语言模型。

模型详细信息

模型描述

这是在Hub上推送的🤗 transformers模型的模型卡片。此模型卡片已自动生成。

开发者： Defog, Inc
模型类型： [文本到SQL]
许可协议： [CC-by-SA-4.0]
微调自模型： [CodeLlama-7B]

模型源（可选）

用途

该模型旨在供非技术用户使用，以理解SQL数据库中的数据。它被视为分析工具，而不是数据库管理工具。

该模型尚未训练以拒绝数据库写访问用户的恶意请求，因此应仅由具有只读访问权限的用户使用。

如何开始使用模型

请使用此代码链接以开始使用模型。

提示

请使用以下提示以获取最佳结果。请记住，为了获得最佳结果，请使用 do_sample=False 和 num_beams=4。

### Task
Generate a SQL query to answer [QUESTION]{user_question}[/QUESTION]

### Database Schema
The query will run on a database with the following schema:
{table_metadata_string_DDL_statements}

### Answer
Given the database schema, here is the SQL query that [QUESTION]{user_question}[/QUESTION]
[SQL]

评估

该模型在Defog开发的基于PostgreSQL的评估框架SQL-Eval上进行了评估，用于测试和校准模型能力。

有关SQLEval背后的方法的更多信息，请参阅此处。

结果

我们将生成的每个问题分类到6个类别之一。表格显示了每个模型在各个类别中回答正确问题的百分比。

	日期	分组	排序	比率	连接	where
sqlcoder-70b	96	91.4	97.1	85.7	97.1	91.4
sqlcoder-7b-2	96	91.4	94.3	91.4	94.3	77.1
sqlcoder-34b	80	94.3	85.7	77.1	85.7	80
gpt-4	72	94.3	97.1	80	91.4	80
gpt-4-turbo	76	91.4	91.4	62.8	88.6	77.1
natural-sql-7b	56	88.6	85.7	60	88.6	80
sqlcoder-7b	64	82.9	74.3	54.3	74.3	74.3
gpt-3.5	72	77.1	82.8	34.3	65.7	71.4
claude-2	52	71.4	74.3	57.1	65.7	62.9

模型卡片联系方式

在X上联系我们 @defogdata 或通过电子邮件 founders@defog.ai

---
license: cc-by-sa-4.0
---
# Update notice
The model weights were updated at 7 AM UTC on Feb 7, 2024. The new model weights lead to a much more performant model – particularly for joins.

If you downloaded the model before that, please redownload the weights for best performance.

# Model Card for SQLCoder-7B-2

A capable large language model for natural language to SQL generation.

![image/png](https://cdn-uploads.huggingface.co/production/uploads/603bbad3fd770a9997b57cb6/AYUE2y14vy2XkD9MZpScu.png)

## Model Details

### Model Description

This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.

- **Developed by:** [Defog, Inc](https://defog.ai)
- **Model type:** [Text to SQL]
- **License:** [CC-by-SA-4.0]
- **Finetuned from model:** [CodeLlama-7B]

### Model Sources [optional]
- [**HuggingFace:**](https://hugging-face.cn/defog/sqlcoder-70b-alpha)
- [**GitHub:**](https://github.com/defog-ai/sqlcoder)
- [**Demo:**](https://defog.ai/sqlcoder-demo/)

## Uses

This model is intended to be used by non-technical users to understand data inside their SQL databases. It is meant as an analytics tool, and not as a database admin tool.

This model has not been trained to reject malicious requests from users with write access to databases, and should only be used by users with read-only access.

## How to Get Started with the Model

Use the code [here](https://github.com/defog-ai/sqlcoder/blob/main/inference.py) to get started with the model.

## Prompt

Please use the following prompt for optimal results. Please remember to use `do_sample=False` and `num_beams=4` for optimal results.

```
### Task
Generate a SQL query to answer [QUESTION]{user_question}[/QUESTION]

### Database Schema
The query will run on a database with the following schema:
{table_metadata_string_DDL_statements}

### Answer
Given the database schema, here is the SQL query that [QUESTION]{user_question}[/QUESTION]
[SQL]
```

## Evaluation

This model was evaluated on [SQL-Eval](https://github.com/defog-ai/sql-eval), a PostgreSQL based evaluation framework developed by Defog for testing and alignment of model capabilities.

You can read more about the methodology behind SQLEval [here](https://defog.ai/blog/open-sourcing-sqleval/).

### Results

We classified each generated question into one of 6 categories. The table displays the percentage of questions answered correctly by each model, broken down by category.

|                | date | group_by | order_by | ratio | join | where |
| -------------- | ---- | -------- | -------- | ----- | ---- | ----- |
| sqlcoder-70b   | 96   | 91.4     | 97.1     | 85.7  | 97.1 | 91.4  |
| sqlcoder-7b-2  | 96   | 91.4     | 94.3     | 91.4  | 94.3 | 77.1  |
| sqlcoder-34b   | 80   | 94.3     | 85.7     | 77.1  | 85.7 | 80    |
| gpt-4          | 72   | 94.3     | 97.1     | 80    | 91.4 | 80    |
| gpt-4-turbo    | 76   | 91.4     | 91.4     | 62.8  | 88.6 | 77.1  |
| natural-sql-7b | 56   | 88.6     | 85.7     | 60    | 88.6 | 80    |
| sqlcoder-7b    | 64   | 82.9     | 74.3     | 54.3  | 74.3 | 74.3  |
| gpt-3.5        | 72   | 77.1     | 82.8     | 34.3  | 65.7 | 71.4  |
| claude-2       | 52   | 71.4     | 74.3     | 57.1  | 65.7 | 62.9  |

## Model Card Contact

Contact us on X at [@defogdata](https://twitter.com/defogdata), or on email at [founders@defog.ai](mailto:founders@defog.ai)

Paste, drop or click to upload images (.png, .jpeg, .jpg, .svg, .gif)