# Database Analysis Script

## Overview
The `analyze_db.py` script is designed to analyze SQLite database files for corruption and integrity issues. It performs various checks including SQLite's built-in integrity check, table structure validation, and attempts to identify specific errors.

## Features
- SQLite integrity check using `PRAGMA integrity_check`
- Foreign key constraint validation
- Database structure analysis (tables, columns, row counts)
- Journal mode and WAL file analysis
- Index validation
- Database page analysis
- Comprehensive data recovery options for corrupted databases:
  - Standard recovery: Creates a new database with the same schema and copies data
  - Advanced recovery techniques (with `--advanced` flag):
    - Dump and restore: Uses SQLite's dump and restore functionality
    - VACUUM recovery: Attempts to repair using SQLite's VACUUM command
    - Table-by-table recovery: Extracts each table individually to recover partial data

## Requirements
- Python 3.6 or higher
- SQLite3 module (included in Python standard library)

## Usage
```bash
# Basic usage (interactive mode)
python src/scripts/analyze_db.py /path/to/your/database.db

# Automatic recovery mode (no prompts)
python src/scripts/analyze_db.py --recovery /path/to/your/database.db

# Advanced recovery mode (tries multiple recovery methods)
python src/scripts/analyze_db.py --recovery --advanced /path/to/your/database.db

# Or if you made the script executable
./src/scripts/analyze_db.py /path/to/your/database.db
./src/scripts/analyze_db.py --recovery /path/to/your/database.db
./src/scripts/analyze_db.py --recovery --advanced /path/to/your/database.db
```

### Using the test script
The `test_analyze_db.py` script provides a convenient way to analyze the project's database:

```bash
# Basic usage (interactive mode)
python src/scripts/test_analyze_db.py

# Automatic recovery mode
python src/scripts/test_analyze_db.py --recovery

# Advanced recovery mode
python src/scripts/test_analyze_db.py --recovery --advanced
```

## Output
The script outputs analysis results to:
1. The console (standard output)
2. A log file named `db_analysis_YYYYMMDD_HHMMSS.log` in the current directory

## Recovery
The script offers multiple recovery methods for corrupted databases:

### Standard Recovery (Default)
When issues are detected, the script will:
1. Create a backup of the original database
2. Create a new database with the same schema
3. Copy as much data as possible from the original to the new database
4. Save the recovered database as `[original_filename].recovered`

### Advanced Recovery (with `--advanced` flag)
When the `--advanced` flag is used, the script will attempt multiple recovery methods:

1. **Standard Recovery** (as described above)
2. **Dump and Restore Recovery**:
   - Uses SQLite's `.dump` command to extract all database content
   - Creates a new database from the dump
   - Saves as `[original_filename].dump_recovered`

3. **VACUUM Recovery**:
   - Creates a copy of the original database
   - Runs the `VACUUM` command to rebuild the database file
   - Saves as `[original_filename].vacuum_recovered`

4. **Table-by-Table Recovery**:
   - Extracts each table individually
   - Copies data row by row to handle partially corrupted tables
   - Saves as `[original_filename].table_recovered`

### Automatic Recovery Mode
With the `--recovery` flag, the script will automatically attempt recovery without prompting for confirmation. This is useful for automated scripts or when you're sure you want to recover the database.

## Examples

### Basic Analysis
```bash
$ ./src/scripts/analyze_db.py data/database/sqlite.db
2023-05-15 10:30:45 - INFO - Starting analysis of database: data/database/sqlite.db
2023-05-15 10:30:45 - INFO - Successfully connected to database: data/database/sqlite.db
2023-05-15 10:30:45 - INFO - Running SQLite integrity check...
2023-05-15 10:30:45 - INFO - Integrity check passed: No corruption detected by SQLite
...
2023-05-15 10:30:46 - INFO - === Analysis Summary ===
2023-05-15 10:30:46 - INFO - Integrity check: PASSED
2023-05-15 10:30:46 - INFO - Foreign key check: PASSED
2023-05-15 10:30:46 - INFO - Structure check: PASSED
2023-05-15 10:30:46 - INFO - Corruption check: PASSED
2023-05-15 10:30:46 - INFO - OVERALL ASSESSMENT: The database appears to be in good condition.
```

### Automatic Recovery
```bash
$ ./src/scripts/analyze_db.py --recovery data/database/corrupted.db
2023-05-15 10:35:12 - INFO - Starting analysis of database: data/database/corrupted.db
2023-05-15 10:35:12 - INFO - Successfully connected to database: data/database/corrupted.db
2023-05-15 10:35:12 - INFO - Running SQLite integrity check...
2023-05-15 10:35:12 - ERROR - Integrity check failed. Issues found:
2023-05-15 10:35:12 - ERROR -   - *** in database main ***
2023-05-15 10:35:12 - WARNING - OVERALL ASSESSMENT: Issues were detected in the database.
2023-05-15 10:35:12 - INFO - Auto-recovery mode enabled. Attempting recovery...
2023-05-15 10:35:12 - INFO - Attempting database recovery operations...
2023-05-15 10:35:12 - INFO - Created backup at: data/database/corrupted.db.backup_20230515_103512
2023-05-15 10:35:12 - INFO - Method 1: Standard recovery - attempting to recover data to: data/database/corrupted.db.recovered
2023-05-15 10:35:15 - INFO - Standard recovery completed. New database created at: data/database/corrupted.db.recovered
2023-05-15 10:35:15 - INFO - At least one recovery method was successful.
2023-05-15 10:35:15 - INFO - Database analysis/recovery completed successfully.
```

### Advanced Recovery
```bash
$ ./src/scripts/analyze_db.py --recovery --advanced data/database/severely_corrupted.db
2023-05-15 10:40:22 - INFO - Starting analysis of database: data/database/severely_corrupted.db
...
2023-05-15 10:40:22 - WARNING - OVERALL ASSESSMENT: Issues were detected in the database.
2023-05-15 10:40:22 - INFO - Auto-recovery mode enabled. Attempting recovery...
2023-05-15 10:40:22 - INFO - Attempting database recovery operations...
2023-05-15 10:40:22 - INFO - Created backup at: data/database/severely_corrupted.db.backup_20230515_104022
2023-05-15 10:40:22 - INFO - Method 1: Standard recovery - attempting to recover data to: data/database/severely_corrupted.db.recovered
2023-05-15 10:40:25 - ERROR - Standard recovery failed: database disk image is malformed
2023-05-15 10:40:25 - INFO - Method 2: Dump and restore recovery - attempting to recover to: data/database/severely_corrupted.db.dump_recovered
2023-05-15 10:40:28 - ERROR - Failed to dump database: Command returned non-zero exit status 1
2023-05-15 10:40:28 - INFO - Method 3: VACUUM recovery - attempting to recover to: data/database/severely_corrupted.db.vacuum_recovered
2023-05-15 10:40:30 - ERROR - VACUUM recovery failed: database disk image is malformed
2023-05-15 10:40:30 - INFO - Method 4: Table-by-table recovery - attempting to recover to: data/database/severely_corrupted.db.table_recovered
2023-05-15 10:40:35 - INFO - Recovered table users with 9 rows
2023-05-15 10:40:35 - INFO - Table-by-table recovery completed with at least one table recovered at data/database/severely_corrupted.db.table_recovered
2023-05-15 10:40:35 - INFO - At least one recovery method was successful.
2023-05-15 10:40:35 - INFO - Database analysis/recovery completed successfully.
```

## Common Issues and Solutions

### "Database file is locked"
This usually happens when another process is using the database. Close any applications that might be using the database and try again.

### "Unable to open database file"
Check file permissions and make sure the path is correct. The user running the script needs read access to the database file.

### "Malformed database"
If the script detects corruption, follow the recovery prompts. After recovery, check the recovered database to ensure critical data was preserved.

## Notes
- Always make a backup of your database before running any recovery operations
- For severely corrupted databases, professional data recovery services might be needed
- This script is a diagnostic tool and may not be able to recover all data from heavily corrupted databases
