| ---
|
| license: mit
|
| language:
|
| - zho
|
| - eng
|
| - fra
|
| - spa
|
| - por
|
| - deu
|
| - ita
|
| - rus
|
| - jpn
|
| - kor
|
| - vie
|
| - tha
|
| - ara
|
| base_model:
|
| - Qwen/Qwen2.5-1.5B-Instruct
|
| pipeline_tag: text2text-generation
|
| library_name: transformers
|
| tags:
|
| - Text-To-SQL
|
| - Arabic
|
| - Spider
|
| - SQL
|
| ---
|
|
|
| # Model Card for Arabic Text-To-SQL (OsamaMo)
|
|
|
| ## Model Details
|
|
|
| ### Model Description
|
|
|
| This model is fine-tuned on the Spider dataset with Arabic-translated questions for the Text-To-SQL task. It is based on **Qwen/Qwen2.5-1.5B-Instruct** and trained using LoRA on Kaggle for 15 hours on a **P100 8GB GPU**.
|
|
|
| - **Developed by:** Osama Mohamed ([OsamaMo](https://huggingface.co/OsamaMo))
|
| - **Funded by:** Self-funded
|
| - **Shared by:** Osama Mohamed
|
| - **Model type:** Text-to-SQL fine-tuned model
|
| - **Language(s):** Arabic (ar)
|
| - **License:** MIT
|
| - **Finetuned from:** Qwen/Qwen2.5-1.5B-Instruct
|
|
|
| ### Model Sources
|
|
|
| - **Repository:** [Hugging Face Model Hub](https://huggingface.co/OsamaMo/Arabic_Text-To-SQL)
|
| - **Dataset:** Spider (translated to Arabic)
|
| - **Training Script:** [LLaMA-Factory](https://github.com/huggingface/transformers/tree/main/src/transformers/models/llama_factory)
|
|
|
| ## Uses
|
|
|
| ### Direct Use
|
|
|
| This model is intended for converting **Arabic natural language questions** into SQL queries. It can be used for database querying in Arabic-speaking applications.
|
|
|
| ### Downstream Use
|
|
|
| Can be fine-tuned further for specific databases or Arabic dialect adaptations.
|
|
|
| ### Out-of-Scope Use
|
|
|
| - The model is **not** intended for direct execution of SQL queries.
|
| - Not recommended for non-database-related NLP tasks.
|
|
|
| ## Bias, Risks, and Limitations
|
|
|
| - The model might generate incorrect or non-optimized SQL queries.
|
| - Bias may exist due to dataset translations and model pretraining data.
|
|
|
| ### Recommendations
|
|
|
| - Validate generated SQL queries before execution.
|
| - Ensure compatibility with specific database schemas.
|
|
|
| ## How to Get Started with the Model
|
| ### Load Model
|
| ```python
|
| from transformers import AutoModelForCausalLM, AutoTokenizer
|
| import torch
|
| import re
|
|
|
| device = "cuda" if torch.cuda.is_available() else "cpu"
|
| base_model_id = "Qwen/Qwen2.5-1.5B-Instruct"
|
| finetuned_model_id = "OsamaMo/Arabic_Text-To-SQL_using_Qwen2.5-1.5B"
|
|
|
| # Load the base model and adapter for fine-tuning
|
| model = AutoModelForCausalLM.from_pretrained(
|
| base_model_id,
|
| device_map="auto",
|
| torch_dtype=torch.bfloat16
|
| )
|
| model.load_adapter(finetuned_model_id)
|
|
|
| tokenizer = AutoTokenizer.from_pretrained(base_model_id)
|
|
|
| def generate_resp(messages):
|
| text = tokenizer.apply_chat_template(
|
| messages,
|
| tokenize=False,
|
| add_generation_prompt=True
|
| )
|
| model_inputs = tokenizer([text], return_tensors="pt").to(device)
|
| generated_ids = model.generate(
|
| model_inputs.input_ids,
|
| max_new_tokens=1024,
|
| do_sample=False, temperature= False,
|
| )
|
| generated_ids = [
|
| output_ids[len(input_ids):]
|
| for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
|
| ]
|
| response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
|
| return response
|
| ```
|
|
|
|
|
|
|
| ### Example Usage
|
| ```python
|
|
|
| # Production-ready system message for SQL generation
|
| system_message = (
|
| "You are a highly advanced Arabic text-to-SQL converter. Your mission is to Understand first the db schema and reltions between it and then accurately transform Arabic "
|
| "natural language queries into SQL queries with precision and clarity.\n"
|
| )
|
|
|
| def get_sql_query(db_schema, arabic_query):
|
| # Construct the instruction message including the DB schema and the Arabic query
|
| instruction_message = "\n".join([
|
| "## DB-Schema:",
|
| db_schema,
|
| "",
|
| "## User-Prompt:",
|
| arabic_query,
|
| "# Output SQL:",
|
| "```SQL"
|
| ])
|
|
|
| messages = [
|
| {"role": "system", "content": system_message},
|
| {"role": "user", "content": instruction_message}
|
| ]
|
|
|
| response = generate_resp(messages)
|
|
|
| # Extract the SQL query from the response using a regex to capture text within the ```sql markdown block
|
| match = re.search(r"```sql\s*(.*?)\s*```", response, re.DOTALL | re.IGNORECASE)
|
| if match:
|
| sql_query = match.group(1).strip()
|
| return sql_query
|
| else:
|
| return response.strip()
|
|
|
| # Example usage:
|
| example_db_schema = r'''{
|
| 'Pharmcy':
|
| CREATE TABLE `purchase` (
|
| `BARCODE` varchar(20) NOT NULL,
|
| `NAME` varchar(50) NOT NULL,
|
| `TYPE` varchar(20) NOT NULL,
|
| `COMPANY_NAME` varchar(20) NOT NULL,
|
| `QUANTITY` int NOT NULL,
|
| `PRICE` double NOT NULL,
|
| `AMOUNT` double NOT NULL,
|
| PRIMARY KEY (`BARCODE`),
|
| KEY `fkr3` (`COMPANY_NAME`),
|
| CONSTRAINT `fkr3` FOREIGN KEY (`COMPANY_NAME`) REFERENCES `company` (`NAME`) ON DELETE CASCADE ON UPDATE CASCADE
|
| ) ENGINE=InnoDB DEFAULT CHARSET=latin1
|
|
|
| CREATE TABLE `sales` (
|
| `BARCODE` varchar(20) NOT NULL,
|
| `NAME` varchar(50) NOT NULL,
|
| `TYPE` varchar(10) NOT NULL,
|
| `DOSE` varchar(10) NOT NULL,
|
| `QUANTITY` int NOT NULL,
|
| `PRICE` double NOT NULL,
|
| `AMOUNT` double NOT NULL,
|
| `DATE` varchar(15) NOT NULL
|
| ) ENGINE=InnoDB DEFAULT CHARSET=latin1
|
|
|
| CREATE TABLE `users` (
|
| `ID` int NOT NULL,
|
| `NAME` varchar(50) NOT NULL,
|
| `DOB` varchar(20) NOT NULL,
|
| `ADDRESS` varchar(100) NOT NULL,
|
| `PHONE` varchar(20) NOT NULL,
|
| `SALARY` double NOT NULL,
|
| `PASSWORD` varchar(20) NOT NULL,
|
| PRIMARY KEY (`ID`)
|
| ) ENGINE=InnoDB DEFAULT CHARSET=latin1
|
|
|
| CREATE TABLE `history_sales` (
|
| `USER_NAME` varchar(20) NOT NULL,
|
| `BARCODE` varchar(20) NOT NULL,
|
| `NAME` varchar(50) NOT NULL,
|
| `TYPE` varchar(10) NOT NULL,
|
| `DOSE` varchar(10) NOT NULL,
|
| `QUANTITY` int NOT NULL,
|
| `PRICE` double NOT NULL,
|
| `AMOUNT` double NOT NULL,
|
| `DATE` varchar(15) NOT NULL,
|
| `TIME` varchar(20) NOT NULL
|
| ) ENGINE=InnoDB DEFAULT CHARSET=latin1
|
|
|
| CREATE TABLE `expiry` (
|
| `PRODUCT_NAME` varchar(50) NOT NULL,
|
| `PRODUCT_CODE` varchar(20) NOT NULL,
|
| `DATE_OF_EXPIRY` varchar(10) NOT NULL,
|
| `QUANTITY_REMAIN` int NOT NULL
|
| ) ENGINE=InnoDB DEFAULT CHARSET=latin1
|
|
|
| CREATE TABLE `drugs` (
|
| `NAME` varchar(50) NOT NULL,
|
| `TYPE` varchar(20) NOT NULL,
|
| `BARCODE` varchar(20) NOT NULL,
|
| `DOSE` varchar(10) NOT NULL,
|
| `CODE` varchar(10) NOT NULL,
|
| `COST_PRICE` double NOT NULL,
|
| `SELLING_PRICE` double NOT NULL,
|
| `EXPIRY` varchar(20) NOT NULL,
|
| `COMPANY_NAME` varchar(50) NOT NULL,
|
| `PRODUCTION_DATE` date NOT NULL,
|
| `EXPIRATION_DATE` date NOT NULL,
|
| `PLACE` varchar(20) NOT NULL,
|
| `QUANTITY` int NOT NULL,
|
| PRIMARY KEY (`BARCODE`)
|
| ) ENGINE=InnoDB DEFAULT CHARSET=latin1
|
|
|
| CREATE TABLE `company` (
|
| `NAME` varchar(50) NOT NULL,
|
| `ADDRESS` varchar(50) NOT NULL,
|
| `PHONE` varchar(20) NOT NULL,
|
| PRIMARY KEY (`NAME`)
|
| ) ENGINE=InnoDB DEFAULT CHARSET=latin1
|
|
|
| Answer the following questions about this schema:
|
| }'''
|
|
|
| example_arabic_query = "اريد الباركود الخاص بدواء يبداء اسمه بحرف 's'"
|
|
|
| sql_result = get_sql_query(example_db_schema, example_arabic_query)
|
| print("استعلام SQL الناتج:")
|
| print(sql_result)
|
| ```
|
|
|
| ## Training Details
|
|
|
| ### Training Data
|
|
|
| - Dataset: **Spider (translated into Arabic)**
|
| - Preprocessing: Questions converted to Arabic while keeping SQL queries unchanged.
|
| - Training format:
|
| - System instruction guiding Arabic-to-SQL conversion.
|
| - Database schema provided for context.
|
| - Arabic user queries mapped to correct SQL output.
|
| - Output is strictly formatted SQL queries enclosed in markdown code blocks.
|
|
|
| ### Training Procedure
|
|
|
| #### Training Hyperparameters
|
|
|
| - **Batch size:** 1 (per device)
|
| - **Gradient accumulation:** 4 steps
|
| - **Learning rate:** 1.0e-4
|
| - **Epochs:** 3
|
| - **Scheduler:** Cosine
|
| - **Warmup ratio:** 0.1
|
| - **Precision:** bf16
|
|
|
| #### Speeds, Sizes, Times
|
|
|
| - **Training time:** 15 hours on **NVIDIA P100 8GB**
|
| - **Checkpointing every:** 500 steps
|
|
|
| ## Evaluation
|
|
|
| ### Testing Data
|
|
|
| - Validation dataset: Spider validation set (translated to Arabic)
|
|
|
| ### Metrics
|
|
|
| - Exact Match (EM) for SQL correctness
|
| - Execution Accuracy (EX) on databases
|
|
|
| ### Results
|
|
|
| - Model achieved **competitive SQL generation accuracy** for Arabic queries.
|
| - Further testing required for robustness.
|
|
|
| ## Environmental Impact
|
|
|
| - **Hardware Type:** NVIDIA Tesla P100 8GB
|
| - **Hours used:** 15
|
| - **Cloud Provider:** Kaggle
|
| - **Carbon Emitted:** Estimated using [ML Impact Calculator](https://mlco2.github.io/impact#compute)
|
|
|
| ## Technical Specifications
|
|
|
| ### Model Architecture and Objective
|
|
|
| - Transformer-based **Qwen2.5-1.5B** architecture.
|
| - Fine-tuned for Text-to-SQL task using LoRA.
|
|
|
| ### Compute Infrastructure
|
|
|
| - **Hardware:** Kaggle P100 GPU (8GB VRAM)
|
| - **Software:** Python, Transformers, LLaMA-Factory, Hugging Face Hub
|
|
|
| ## Citation
|
|
|
| If you use this model, please cite:
|
|
|
| ```bibtex
|
| @misc{OsamaMo_ArabicSQL,
|
| author = {Osama Mohamed},
|
| title = {Arabic Text-To-SQL Model},
|
| year = {2024},
|
| howpublished = {\url{https://huggingface.co/OsamaMo/Arabic_Text-To-SQL}}
|
| }
|
| ```
|
|
|
| ## Model Card Contact
|
|
|
| For questions, contact **Osama Mohamed** via Hugging Face ([OsamaMo](https://huggingface.co/OsamaMo)).
|
|
|
|
|
