pxlksr / defog_sqlcoder-7b-2

一种强大的自然语言到SQL生成的大规模语言模型。

1,043 拉取更新于 5 个月前

更新于 5 个月前

5个月前

f0eea48100a6 · 3.9GB

README

许可证: cc-by-sa-4.0

更新说明

模型权重于2024年2月7日UTC 早上7点更新。新的模型权重导致模型性能有了大幅提升——尤其是对于连接操作。

如果您在此之前已下载了模型，请重新下载权重以获得最佳性能。

SQLCoder-7B-2 模型卡片

一种强大的自然语言到SQL生成的大规模语言模型。

模型详细信息

模型描述

这是在Hub上推送的🤗 transformers模型的项目卡片。此项目卡片已自动生成。

开发者： Defog, Inc
模型类型： [文本到SQL]
许可证： [CC-by-SA-4.0]
微调自模型： [CodeLlama-7B]

模型来源 [可选]

用途

此模型旨在供非技术用户使用，以了解他们SQL数据库中的数据。这是一个分析工具，而不是数据库管理员工具。

此模型未经过训练以拒绝对数据库有写访问权限的用户提出的恶意请求，并且仅应由具有只读访问权限的用户使用。

如何开始使用模型

使用此代码 here 开始使用该模型。

提示

请使用以下提示以获取最佳结果。请记住，为了最佳结果，请使用 do_sample=False 和 num_beams=4。

### Task
Generate a SQL query to answer [QUESTION]{user_question}[/QUESTION]

### Database Schema
The query will run on a database with the following schema:
{table_metadata_string_DDL_statements}

### Answer
Given the database schema, here is the SQL query that [QUESTION]{user_question}[/QUESTION]
[SQL]

评估

此模型在SQL-Eval（Defog开发的一个基于PostgreSQL的评估框架，用于测试和调整模型功能）上进行了评估。

您可以在这里了解更多关于SQLEval背后的方法。

结果

我们将生成的每个问题分类为6个类别之一。表显示了每个类别中每个模型正确回答问题的百分比。

	日期	按group_by分组	按order_by排序	比例	连接	where条件
sqlcoder-70b	96	91.4	97.1	85.7	97.1	91.4
sqlcoder-7b-2	96	91.4	94.3	91.4	94.3	77.1
sqlcoder-34b	80	94.3	85.7	77.1	85.7	80
gpt-4	72	94.3	97.1	80	91.4	80
gpt-4-turbo	76	91.4	91.4	62.8	88.6	77.1
natural-sql-7b	56	88.6	85.7	60	88.6	80
sqlcoder-7b	64	82.9	74.3	54.3	74.3	74.3
gpt-3.5	72	77.1	82.8	34.3	65.7	71.4
claude-2	52	71.4	74.3	57.1	65.7	62.9

模型卡片联系方式

在X上通过 @defogdata 或电子邮件 founders@defog.ai 联系我们。

---
license: cc-by-sa-4.0
---
# Update notice
The model weights were updated at 7 AM UTC on Feb 7, 2024. The new model weights lead to a much more performant model – particularly for joins.

If you downloaded the model before that, please redownload the weights for best performance.

# Model Card for SQLCoder-7B-2

A capable large language model for natural language to SQL generation.

![image/png](https://cdn-uploads.huggingface.co/production/uploads/603bbad3fd770a9997b57cb6/AYUE2y14vy2XkD9MZpScu.png)

## Model Details

### Model Description

This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.

- **Developed by:** [Defog, Inc](https://defog.ai)
- **Model type:** [Text to SQL]
- **License:** [CC-by-SA-4.0]
- **Finetuned from model:** [CodeLlama-7B]

### Model Sources [optional]
- [**HuggingFace:**](https://hugging-face.cn/defog/sqlcoder-70b-alpha)
- [**GitHub:**](https://github.com/defog-ai/sqlcoder)
- [**Demo:**](https://defog.ai/sqlcoder-demo/)

## Uses

This model is intended to be used by non-technical users to understand data inside their SQL databases. It is meant as an analytics tool, and not as a database admin tool.

This model has not been trained to reject malicious requests from users with write access to databases, and should only be used by users with read-only access.

## How to Get Started with the Model

Use the code [here](https://github.com/defog-ai/sqlcoder/blob/main/inference.py) to get started with the model.

## Prompt

Please use the following prompt for optimal results. Please remember to use `do_sample=False` and `num_beams=4` for optimal results.

```
### Task
Generate a SQL query to answer [QUESTION]{user_question}[/QUESTION]

### Database Schema
The query will run on a database with the following schema:
{table_metadata_string_DDL_statements}

### Answer
Given the database schema, here is the SQL query that [QUESTION]{user_question}[/QUESTION]
[SQL]
```

## Evaluation

This model was evaluated on [SQL-Eval](https://github.com/defog-ai/sql-eval), a PostgreSQL based evaluation framework developed by Defog for testing and alignment of model capabilities.

You can read more about the methodology behind SQLEval [here](https://defog.ai/blog/open-sourcing-sqleval/).

### Results

We classified each generated question into one of 6 categories. The table displays the percentage of questions answered correctly by each model, broken down by category.

|                | date | group_by | order_by | ratio | join | where |
| -------------- | ---- | -------- | -------- | ----- | ---- | ----- |
| sqlcoder-70b   | 96   | 91.4     | 97.1     | 85.7  | 97.1 | 91.4  |
| sqlcoder-7b-2  | 96   | 91.4     | 94.3     | 91.4  | 94.3 | 77.1  |
| sqlcoder-34b   | 80   | 94.3     | 85.7     | 77.1  | 85.7 | 80    |
| gpt-4          | 72   | 94.3     | 97.1     | 80    | 91.4 | 80    |
| gpt-4-turbo    | 76   | 91.4     | 91.4     | 62.8  | 88.6 | 77.1  |
| natural-sql-7b | 56   | 88.6     | 85.7     | 60    | 88.6 | 80    |
| sqlcoder-7b    | 64   | 82.9     | 74.3     | 54.3  | 74.3 | 74.3  |
| gpt-3.5        | 72   | 77.1     | 82.8     | 34.3  | 65.7 | 71.4  |
| claude-2       | 52   | 71.4     | 74.3     | 57.1  | 65.7 | 62.9  |

## Model Card Contact

Contact us on X at [@defogdata](https://twitter.com/defogdata), or on email at [founders@defog.ai](mailto:founders@defog.ai)

Paste, drop or click to upload images (.png, .jpeg, .jpg, .svg, .gif)