# JSON Business Rules Engine (JSON-BRE)

A Python-based Business Rules Engine designed for financial/credit assessment applications. This engine evaluates loan applications against configurable business rules and policies to determine eligibility and risk assessment.

## Table of Contents

- [Overview](#overview)
- [Architecture](#architecture)
- [Project Structure](#project-structure)
- [Prerequisites](#prerequisites)
- [Installation & Setup](#installation--setup)
- [Configuration](#configuration)
- [API Usage](#api-usage)
- [Rule Implementation](#rule-implementation)
- [Docker Deployment](#docker-deployment)
- [Development](#development)

## Overview

The JSON Business Rules Engine is a flexible, configuration-driven system that processes loan applications through a series of business rules. It supports multiple data sources including MySQL databases, AWS S3 storage, and direct JSON input, making it suitable for complex financial assessment workflows.

### Key Features

- **Multi-source Data Integration**: MySQL, AWS S3, and direct JSON input
- **Configurable Rule Engine**: JSON-driven rule definitions with priority-based execution
- **Fallback Mechanisms**: Automatic fallback rules for error handling
- **Custom Functions**: Dynamic Python function execution for complex business logic
- **Grade-based Evaluation**: Support for multiple evaluation grades (grade-1, grade-2)
- **Comprehensive Credit Assessment**: DPD analysis, bureau data processing, surrogate rules

## Architecture

```
┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│   Flask API     │    │  Rule Engine    │    │  Data Sources   │
│   (index.py)    │◄──►│  (parse_json)   │◄──►│  MySQL + S3     │
└─────────────────┘    └─────────────────┘    └─────────────────┘
         │                       │                       │
         ▼                       ▼                       ▼
┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│  Rule Modules   │    │  Custom Funcs   │    │  Configuration  │
│  (tests/*.py)   │    │  (functions)    │    │  (config.json)  │
└─────────────────┘    └─────────────────┘    └─────────────────┘
```

## Project Structure

```
json-bre-engine/
├── index.py                          # Main Flask application
├── config.json                       # Main configuration file
├── rule.json                         # Sample rule configuration
├── bre_config_exhaustive_list.json   # Comprehensive rule definitions
├── rejection_policy.json             # Rejection policy rules
├── requirements.txt                  # Python dependencies
├── Dockerfile                        # Docker configuration
├── storage.py                        # S3 storage utilities
├── search.py                         # JSON search utilities
├── utils.py                          # Utility functions
├── database.py                       # Database connection and query operations
├── dynamic_functions.py              # Runtime function registry
├── helper_methods.py                 # Helper functions for rule processing
├── autoload.py                       # Configuration autoloading
├── email_test.py                     # Email testing utilities
└── tests/                            # Rule implementation modules
    ├── account_status.py             # Account status validation
    ├── dpd_equifax.py               # Equifax DPD analysis
    ├── dpd_experian.py              # Experian DPD analysis
    ├── function_converter.py        # Data conversion utilities
    ├── keywords_rule.py             # Keyword-based rules
    ├── nach_bounce.py               # NACH bounce analysis
    ├── principal_write_off_amount.py # Principal write-off validation
    └── total_past_dues.py           # Total past dues calculation
```

## Prerequisites

- Python 3.9+
- MySQL 5.7+ or 8.0+
- AWS Account (for S3 integration)
- Git

## Local Machine Setup

### Prerequisites

Before setting up the application, ensure you have the following installed:

- **Python 3.9+** - Download from [python.org](https://www.python.org/downloads/)
- **MySQL 5.7+ or 8.0+** - Download from [mysql.com](https://dev.mysql.com/downloads/)
- **Git** - Download from [git-scm.com](https://git-scm.com/downloads)
- **VS Code** (Recommended) - Download from [code.visualstudio.com](https://code.visualstudio.com/)

### 1. Clone the Repository

```bash
git clone <repository-url>
cd json-bre-engine
```

### 2. VS Code Setup

#### Install Recommended Extensions

Open VS Code and install these recommended extensions:

1. **Python** (ms-python.python) - Python language support
2. **Python Debugger** (ms-python.debugpy) - Debugging support
3. **MySQL** (formulahendry.vscode-mysql) - MySQL database support
4. **Python Docstring Generator** (njpwerner.autodocstring) - Auto-generate docstrings
5. **Python Indent** (KevinRose.vsc-python-indent) - Proper Python indentation

#### Configure VS Code Settings

Create `.vscode/settings.json` in your project root:

```json
{
    "python.defaultInterpreterPath": "./venv/bin/python",
    "python.terminal.activateEnvironment": true,
    "python.linting.enabled": true,
    "python.linting.pylintEnabled": true,
    "python.formatting.provider": "black",
    "python.testing.pytestEnabled": true,
    "python.testing.unittestEnabled": false,
    "files.exclude": {
        "**/__pycache__": true,
        "**/*.pyc": true
    }
}
```

#### Configure Debug Launch Settings

Create `.vscode/launch.json` in your project root:

```json
{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Python: JSON BRE Engine",
            "type": "debugpy",
            "request": "launch",
            "program": "${workspaceFolder}/index.py",
            "console": "integratedTerminal",
            "envFile": "${workspaceFolder}/.env",
            "cwd": "${workspaceFolder}",
            "args": [],
            "justMyCode": true,
            "stopOnEntry": false,
            "python": "${workspaceFolder}/venv/bin/python"
        },
        {
            "name": "Python: JSON BRE Engine (Debug Mode)",
            "type": "debugpy",
            "request": "launch",
            "program": "${workspaceFolder}/index.py",
            "console": "integratedTerminal",
            "envFile": "${workspaceFolder}/.env",
            "cwd": "${workspaceFolder}",
            "args": [],
            "justMyCode": false,
            "stopOnEntry": false,
            "python": "${workspaceFolder}/venv/bin/python",
            "env": {
                "FLASK_DEBUG": "1"
            }
        }
    ]
}
```

### 3. Create Virtual Environment

```bash
# Create virtual environment
python -m venv venv

# Activate virtual environment
# On Windows:
venv\Scripts\activate
# On macOS/Linux:
source venv/bin/activate
```

### 4. Install Dependencies

```bash
pip install -r requirements.txt
```

### 5. Environment Configuration

Create a `.env` file in the project root:

```env
# Database Configuration
DB_HOST=localhost
DB_PORT=3306
DB_USER=your_mysql_username
DB_PASSWORD=your_mysql_password
DB_DATABASE=your_database_name

# AWS S3 Configuration
AWS_ACCESS_KEY_ID=your_aws_access_key
AWS_SECRET_ACCESS_KEY=your_aws_secret_key
AWS_BUCKET_NAME=your_s3_bucket_name
```

### 7. Running the Application in VS Code

#### Using VS Code Debugger

1. **Create Launch Configuration:**
   Create `.vscode/launch.json`:
```json
{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Python: JSON BRE Engine",
            "type": "python",
            "request": "launch",
            "program": "${workspaceFolder}/index.py",
            "console": "integratedTerminal",
            "envFile": "${workspaceFolder}/.env",
            "cwd": "${workspaceFolder}"
        }
    ]
}
```

2. **Start Debugging:**
   - Press `F5` or go to `Run > Start Debugging`
   - The application will start with debugging enabled
   - You can set breakpoints by clicking in the left margin of the code editor

### 8. Testing the Application

#### Test Individual Rule Modules

```bash
# Test account status validation
python tests/account_status.py

# Test DPD analysis
python tests/dpd_equifax.py
python tests/dpd_rejection.py
```

#### Test API Endpoint

Use a tool like Postman or curl to test the API:

```bash
curl -X POST http://localhost:8888/ \
  -H "Content-Type: application/json" \
  -d '{
    "identifier": "12345",
    "rules": {
      "grade-1": [
        {
          "key": "credit_score",
          "value": {
            "source": [{"type": "key"}],
            "against": [
              {
                "type": "db",
                "details": {
                  "value": "sourceFile.cf.credit_report.Score",
                  "primaryColumn": "application_id",
                  "condition": "greaterThanEqual"
                }
              }
            ]
          },
          "priority": 4
        }
      ]
    }
  }'
```

### 9. Troubleshooting VS Code Issues

#### Python Interpreter Issues
- Press `Ctrl+Shift+P` and type "Python: Select Interpreter"
- Choose the interpreter from your virtual environment (`./venv/bin/python`)

#### Extension Issues
- Reload VS Code window: `Ctrl+Shift+P` → "Developer: Reload Window"
- Check extension status in the Extensions panel

#### Database Connection Issues
- Verify MySQL is running: `sudo systemctl status mysql` (Linux) or check Services (Windows)
- Test connection: `mysql -u your_username -p -h localhost`

## Configuration

### Main Configuration (`config.json`)

The main configuration file defines:

1. **Database Sources**:
```json
{
  "sources": {
    "sourceFile": {
      "cf": "cfwebbeta",
      "loc": "line_of_credit"
    },
    "connections": [...]
  }
}
```

2. **S3 File Configuration**:
```json
{
  "autoload": {
    "config": {
      "s3": {
        "AWS_ACCESS_KEY_ID": "::getENV",
        "AWS_SECRET_ACCESS_KEY": "::getENV",
        "AWS_BUCKET_NAME": "::getENV"
      },
      "files": {
        "bureau": {
          "type": "s3",
          "misc": [...]
        }
      }
    }
  }
}
```

3. **Custom Functions**:
```json
{
  "functions": {
    "lang": [
      {
        "python": [
          {
            "function_name()": "def function_name(): ..."
          }
        ]
      }
    ]
  }
}
```

### Rule Configuration

Rules are defined in JSON format with the following structure:

```json
{
  "key": "rule_name",
  "description": ["Rule description"],
  "value": {
    "source": [
      {
        "type": "db|s3|text|key",
        "details": {...}
      }
    ],
    "against": [
      {
        "type": "db|s3|text|list",
        "details": {
          "condition": "equal|greaterThan|lessThan|inList|etc"
        }
      }
    ]
  },
  "priority": 1,
  "fallback": {
    "rule": "fallback_rule_name"
  }
}
```

## API Usage

### Endpoint

```
POST http://localhost:8888/
```

### Request Format

```json
{
  "identifier": "12345",
  "rules": {
    "grade-1": [
      {
        "key": "credit_score",
        "value": {
          "source": [{"type": "key"}],
          "against": [
            {
              "type": "db",
              "details": {
                "value": "sourceFile.cf.credit_report.Score",
                "primaryColumn": "application_id",
                "condition": "greaterThanEqual"
              }
            }
          ]
        },
        "priority": 4
      }
    ]
  },
  "functions": {...},
  "source": {...}
}
```

### Response Format

```json
{
  "grade-1": {
    "policy_status": true,
    "rules": [
      {
        "key": "credit_score",
        "result": true,
        "reason": "Passed: 750 greaterThanEqual 700",
        "source": 750,
        "against": [700, "greaterThanEqual"]
      }
    ],
    "failed_condition": null
  },
  "status": true
}
```

## Database Module

The `database.py` module handles all database-related operations, providing a clean separation of concerns from the main application logic.

### Features

- **Connection Management**: Centralized MySQL connection handling with automatic reconnection
- **Query Execution**: Safe query execution with proper error handling and cursor management
- **Environment Configuration**: Database credentials loaded from environment variables

### Usage

```python
import database

# Get or create a connection
conn = database.get_connection()

# Execute a query
results = database.execute_select_query(conn, "SELECT * FROM table WHERE id = 1")

# Close connection when done
database.close_connection()
```

### Database Configuration

Database connection settings are configured via environment variables in `.env`:

```env
DB_HOST=localhost
DB_PORT=3306
DB_USER=your_mysql_username
DB_PASSWORD=your_mysql_password
DB_DATABASE=your_database_name
```

### Functions

- `create_connection()`: Creates a new MySQL database connection
- `get_connection()`: Gets the current connection or creates a new one
- `set_connection(conn)`: Sets the global database connection
- `close_connection()`: Closes the current database connection
- `execute_select_query(connection, query)`: Executes a SELECT query and returns results
- `init_connection(data)`: Initializes connection (for backward compatibility)

## Rule Implementation

### Rule Types

1. **Database Rules**: Query data from MySQL tables
2. **S3 Rules**: Fetch data from AWS S3 files
3. **Text Rules**: Static text values
4. **Key Rules**: Values from request payload

### Comparison Operators

- `equal` - Exact match
- `greaterThan` - Greater than comparison
- `lessThan` - Less than comparison
- `greaterThanEqual` - Greater than or equal
- `lessThanEqual` - Less than or equal
- `inBetween` - Range check
- `inList` - Check if value exists in list
- `notInList` - Check if value doesn't exist in list

### Custom Functions

Custom functions are defined in the configuration and executed dynamically:

```python
def dateDiffInYear(date1, date2):
    from datetime import datetime
    date1 = datetime.strptime(date1, "%Y-%m-%d")
    date2 = datetime.strptime(date2, "%Y-%m-%d")
    delta = date2 - date1
    return int(delta.days / 365)
```

### Rule Modules

Rule implementation modules in the `tests/` directory handle specific business logic:

- **`account_status.py`**: Account status validation for credit bureau data
- **`dpd_equifax.py`**: Days Past Due analysis for Equifax data
- **`dpd_experian.py`**: Days Past Due analysis for Experian data
- **`keywords_rule.py`**: Keyword-based rule validation
- **`nach_bounce.py`**: NACH bounce analysis
- **`principal_write_off_amount.py`**: Principal write-off validation
- **`total_past_dues.py`**: Total past dues calculation

## Docker Deployment

### Build Docker Image

```bash
docker build -t json-bre-engine .
```

### Run Container

```bash
docker run -p 8888:8888 --env-file .env json-bre-engine
```

## Development

### Adding New Rules

1. **Define Rule in Configuration**:
```json
{
  "key": "new_rule",
  "value": {
    "source": [...],
    "against": [...]
  },
  "priority": 1
}
```

2. **Implement Custom Function** (if needed):
```python
def new_rule_function(param1, param2):
    # Implementation
    return result
```

3. **Add to Rule Module** (if complex logic):
```python
def check_new_rule(rule_data, input_data):
    # Complex rule implementation
    return result
```

### Testing

Test individual rule modules:

```bash
python tests/account_status.py
python tests/dpd_equifax.py
```



## License

[Add your license information here]

## Contributing

[Add contribution guidelines here]

## Support

[Add support information here]