| --- |
| license: mit |
| language: |
| - ar |
| base_model: |
| - Qwen/Qwen2.5-1.5B-Instruct |
| pipeline_tag: text2text-generation |
| library_name: transformers |
| tags: |
| - Text-To-SQL |
| - Arabic |
| - Spider |
| - SQL |
| --- |
| |
| # Model Card for Arabic Text-To-SQL (OsamaMo) |
|
|
| ## Model Details |
|
|
| ### Model Description |
|
|
| This model is fine-tuned on the Spider dataset with Arabic-translated questions for the Text-To-SQL task. It is based on **Qwen/Qwen2.5-1.5B-Instruct** and trained using LoRA on Kaggle for 15 hours on a **P100 8GB GPU**. |
|
|
| - **Developed by:** Osama Mohamed ([OsamaMo](https://huggingface.co/OsamaMo)) |
| - **Funded by:** Self-funded |
| - **Shared by:** Osama Mohamed |
| - **Model type:** Text-to-SQL fine-tuned model |
| - **Language(s):** Arabic (ar) |
| - **License:** MIT |
| - **Finetuned from:** Qwen/Qwen2.5-1.5B-Instruct |
|
|
| ### Model Sources |
|
|
| - **Repository:** [Hugging Face Model Hub](https://huggingface.co/OsamaMo/Arabic_Text-To-SQL) |
| - **Dataset:** Spider (translated to Arabic) |
| - **Training Script:** [LLaMA-Factory](https://github.com/huggingface/transformers/tree/main/src/transformers/models/llama_factory) |
|
|
| ## Uses |
|
|
| ### Direct Use |
|
|
| This model is intended for converting **Arabic natural language questions** into SQL queries. It can be used for database querying in Arabic-speaking applications. |
|
|
| ### Downstream Use |
|
|
| Can be fine-tuned further for specific databases or Arabic dialect adaptations. |
|
|
| ### Out-of-Scope Use |
|
|
| - The model is **not** intended for direct execution of SQL queries. |
| - Not recommended for non-database-related NLP tasks. |
|
|
| ## Bias, Risks, and Limitations |
|
|
| - The model might generate incorrect or non-optimized SQL queries. |
| - Bias may exist due to dataset translations and model pretraining data. |
|
|
| ### Recommendations |
|
|
| - Validate generated SQL queries before execution. |
| - Ensure compatibility with specific database schemas. |
|
|
| ## How to Get Started with the Model |
| ### Load Model |
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| import torch |
| import re |
| |
| device = "cuda" if torch.cuda.is_available() else "cpu" |
| base_model_id = "Qwen/Qwen2.5-1.5B-Instruct" |
| finetuned_model_id = "OsamaMo/Arabic_Text-To-SQL_using_Qwen2.5-1.5B" |
| |
| # Load the base model and adapter for fine-tuning |
| model = AutoModelForCausalLM.from_pretrained( |
| base_model_id, |
| device_map="auto", |
| torch_dtype=torch.bfloat16 |
| ) |
| model.load_adapter(finetuned_model_id) |
| |
| tokenizer = AutoTokenizer.from_pretrained(base_model_id) |
| |
| def generate_resp(messages): |
| text = tokenizer.apply_chat_template( |
| messages, |
| tokenize=False, |
| add_generation_prompt=True |
| ) |
| model_inputs = tokenizer([text], return_tensors="pt").to(device) |
| generated_ids = model.generate( |
| model_inputs.input_ids, |
| max_new_tokens=1024, |
| do_sample=False, temperature= False, |
| ) |
| generated_ids = [ |
| output_ids[len(input_ids):] |
| for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids) |
| ] |
| response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0] |
| return response |
| ``` |
|
|
|
|
|
|
| ### Example Usage |
| ```python |
| |
| # Production-ready system message for SQL generation |
| system_message = ( |
| "You are a highly advanced Arabic text-to-SQL converter. Your mission is to Understand first the db schema and reltions between it and then accurately transform Arabic " |
| "natural language queries into SQL queries with precision and clarity.\n" |
| ) |
| |
| def get_sql_query(db_schema, arabic_query): |
| # Construct the instruction message including the DB schema and the Arabic query |
| instruction_message = "\n".join([ |
| "## DB-Schema:", |
| db_schema, |
| "", |
| "## User-Prompt:", |
| arabic_query, |
| "# Output SQL:", |
| "```SQL" |
| ]) |
| |
| messages = [ |
| {"role": "system", "content": system_message}, |
| {"role": "user", "content": instruction_message} |
| ] |
| |
| response = generate_resp(messages) |
| |
| # Extract the SQL query from the response using a regex to capture text within the ```sql markdown block |
| match = re.search(r"```sql\s*(.*?)\s*```", response, re.DOTALL | re.IGNORECASE) |
| if match: |
| sql_query = match.group(1).strip() |
| return sql_query |
| else: |
| return response.strip() |
| |
| # Example usage: |
| example_db_schema = r'''{ |
| 'Pharmcy': |
| CREATE TABLE `purchase` ( |
| `BARCODE` varchar(20) NOT NULL, |
| `NAME` varchar(50) NOT NULL, |
| `TYPE` varchar(20) NOT NULL, |
| `COMPANY_NAME` varchar(20) NOT NULL, |
| `QUANTITY` int NOT NULL, |
| `PRICE` double NOT NULL, |
| `AMOUNT` double NOT NULL, |
| PRIMARY KEY (`BARCODE`), |
| KEY `fkr3` (`COMPANY_NAME`), |
| CONSTRAINT `fkr3` FOREIGN KEY (`COMPANY_NAME`) REFERENCES `company` (`NAME`) ON DELETE CASCADE ON UPDATE CASCADE |
| ) ENGINE=InnoDB DEFAULT CHARSET=latin1 |
| |
| CREATE TABLE `sales` ( |
| `BARCODE` varchar(20) NOT NULL, |
| `NAME` varchar(50) NOT NULL, |
| `TYPE` varchar(10) NOT NULL, |
| `DOSE` varchar(10) NOT NULL, |
| `QUANTITY` int NOT NULL, |
| `PRICE` double NOT NULL, |
| `AMOUNT` double NOT NULL, |
| `DATE` varchar(15) NOT NULL |
| ) ENGINE=InnoDB DEFAULT CHARSET=latin1 |
| |
| CREATE TABLE `users` ( |
| `ID` int NOT NULL, |
| `NAME` varchar(50) NOT NULL, |
| `DOB` varchar(20) NOT NULL, |
| `ADDRESS` varchar(100) NOT NULL, |
| `PHONE` varchar(20) NOT NULL, |
| `SALARY` double NOT NULL, |
| `PASSWORD` varchar(20) NOT NULL, |
| PRIMARY KEY (`ID`) |
| ) ENGINE=InnoDB DEFAULT CHARSET=latin1 |
| |
| CREATE TABLE `history_sales` ( |
| `USER_NAME` varchar(20) NOT NULL, |
| `BARCODE` varchar(20) NOT NULL, |
| `NAME` varchar(50) NOT NULL, |
| `TYPE` varchar(10) NOT NULL, |
| `DOSE` varchar(10) NOT NULL, |
| `QUANTITY` int NOT NULL, |
| `PRICE` double NOT NULL, |
| `AMOUNT` double NOT NULL, |
| `DATE` varchar(15) NOT NULL, |
| `TIME` varchar(20) NOT NULL |
| ) ENGINE=InnoDB DEFAULT CHARSET=latin1 |
| |
| CREATE TABLE `expiry` ( |
| `PRODUCT_NAME` varchar(50) NOT NULL, |
| `PRODUCT_CODE` varchar(20) NOT NULL, |
| `DATE_OF_EXPIRY` varchar(10) NOT NULL, |
| `QUANTITY_REMAIN` int NOT NULL |
| ) ENGINE=InnoDB DEFAULT CHARSET=latin1 |
| |
| CREATE TABLE `drugs` ( |
| `NAME` varchar(50) NOT NULL, |
| `TYPE` varchar(20) NOT NULL, |
| `BARCODE` varchar(20) NOT NULL, |
| `DOSE` varchar(10) NOT NULL, |
| `CODE` varchar(10) NOT NULL, |
| `COST_PRICE` double NOT NULL, |
| `SELLING_PRICE` double NOT NULL, |
| `EXPIRY` varchar(20) NOT NULL, |
| `COMPANY_NAME` varchar(50) NOT NULL, |
| `PRODUCTION_DATE` date NOT NULL, |
| `EXPIRATION_DATE` date NOT NULL, |
| `PLACE` varchar(20) NOT NULL, |
| `QUANTITY` int NOT NULL, |
| PRIMARY KEY (`BARCODE`) |
| ) ENGINE=InnoDB DEFAULT CHARSET=latin1 |
| |
| CREATE TABLE `company` ( |
| `NAME` varchar(50) NOT NULL, |
| `ADDRESS` varchar(50) NOT NULL, |
| `PHONE` varchar(20) NOT NULL, |
| PRIMARY KEY (`NAME`) |
| ) ENGINE=InnoDB DEFAULT CHARSET=latin1 |
| |
| Answer the following questions about this schema: |
| }''' |
| |
| example_arabic_query = "اريد الباركود الخاص بدواء يبداء اسمه بحرف 's'" |
|
|
| sql_result = get_sql_query(example_db_schema, example_arabic_query) |
| print("استعلام SQL الناتج:") |
| print(sql_result) |
| ``` |
| |
| ## Training Details |
| |
| ### Training Data |
| |
| - Dataset: **Spider (translated into Arabic)** |
| - Preprocessing: Questions converted to Arabic while keeping SQL queries unchanged. |
| - Training format: |
| - System instruction guiding Arabic-to-SQL conversion. |
| - Database schema provided for context. |
| - Arabic user queries mapped to correct SQL output. |
| - Output is strictly formatted SQL queries enclosed in markdown code blocks. |
| |
| ### Training Procedure |
| |
| #### Training Hyperparameters |
| |
| - **Batch size:** 1 (per device) |
| - **Gradient accumulation:** 4 steps |
| - **Learning rate:** 1.0e-4 |
| - **Epochs:** 3 |
| - **Scheduler:** Cosine |
| - **Warmup ratio:** 0.1 |
| - **Precision:** bf16 |
| |
| #### Speeds, Sizes, Times |
| |
| - **Training time:** 15 hours on **NVIDIA P100 8GB** |
| - **Checkpointing every:** 500 steps |
| |
| ## Evaluation |
| |
| ### Testing Data |
| |
| - Validation dataset: Spider validation set (translated to Arabic) |
| |
| ### Metrics |
| |
| - Exact Match (EM) for SQL correctness |
| - Execution Accuracy (EX) on databases |
| |
| ### Results |
| |
| - Model achieved **competitive SQL generation accuracy** for Arabic queries. |
| - Further testing required for robustness. |
| |
| ## Environmental Impact |
| |
| - **Hardware Type:** NVIDIA Tesla P100 8GB |
| - **Hours used:** 15 |
| - **Cloud Provider:** Kaggle |
| - **Carbon Emitted:** Estimated using [ML Impact Calculator](https://mlco2.github.io/impact#compute) |
| |
| ## Technical Specifications |
| |
| ### Model Architecture and Objective |
| |
| - Transformer-based **Qwen2.5-1.5B** architecture. |
| - Fine-tuned for Text-to-SQL task using LoRA. |
| |
| ### Compute Infrastructure |
| |
| - **Hardware:** Kaggle P100 GPU (8GB VRAM) |
| - **Software:** Python, Transformers, LLaMA-Factory, Hugging Face Hub |
| |
| ## Citation |
| |
| If you use this model, please cite: |
| |
| ```bibtex |
| @misc{OsamaMo_ArabicSQL, |
| author = {Osama Mohamed}, |
| title = {Arabic Text-To-SQL Model}, |
| year = {2024}, |
| howpublished = {\url{https://huggingface.co/OsamaMo/Arabic_Text-To-SQL}} |
| } |
| ``` |
| |
| ## Model Card Contact |
| |
| For questions, contact **Osama Mohamed** via Hugging Face ([OsamaMo](https://huggingface.co/OsamaMo)). |
| |
| |
