Why Exasol, feature by feature
Credora uses Exasol as the place where a bank's scattered data meets, is analysed and becomes a decision. This page lists every Exasol feature the project uses, where it is in the code, and why it is the right tool for that job.
| # | Exasol feature | Where | What it does in Credora | Why it matters |
|---|---|---|---|---|
| 1 | Virtual schema: Oracle (Java adapter, JDBC) | 03_virtual_schemas.sql | VS_CBS reads 850,000 core-banking transactions in Oracle in place |
Core banking is never copied. Banks keep their system of record where it is. |
| 2 | Virtual schema: adbc-vs-rs (Rust, ADBC, Arrow) | same | VS_CIF on MySQL and VS_LOS on PostgreSQL |
One adapter for two engines, moving Arrow record batches, not rows |
| 3 | Custom script language container | exasol_setup.py install_rust_slc |
Rust SLC 0.23.1 uploaded to BucketFS and registered as RUST |
Runs the Rust adapter at the SDK version it was built for |
| 4 | BucketFS | exasol_setup.py, 05_ml_udfs.sql |
Holds the SLC, adapter library, ADBC drivers (fetched on first use), the Oracle JDBC driver, and every trained model version | One distributed file store for code, drivers and models |
| 5 | Python UDFs calling APIs (SET scripts) | 04_api_udfs.sql | CIBIL, GSTN and Account Aggregator data fetched by 16 parallel UDF instances through bulk endpoints: 3,490 CIBIL reports in under 1 second | External data comes in with SQL, in parallel, and lands next to the bank data |
| 6 | CONNECTION objects | 02_connections.sql | API addresses, tokens and database credentials | Secrets never appear in SQL text, and access is granted per role |
| 7 | ML training inside the database | TRAIN_PD in 05_ml_udfs.sql |
Gradient-boosted trees with monotonic credit constraints, trained on 5,000 past loans read live from PostgreSQL, measured on a hold-out set (AUC, Gini, permutation importance) | No data export to a separate ML platform |
| 8 | ML scoring with explanations | PD_SCORE |
Probability of default for 1,000 companies in about 2 seconds, plus the effect of each feature on each company | Every decision can say why |
| 9 | Lua scripting | 06_run_analysis.sql | RUN_ANALYSIS runs 11 logged steps; TRAIN_MODEL versions models |
The whole pipeline is one database call, with a timing for each step |
| 10 | Analytic SQL: window functions, CTEs, pushdown aggregation | step 7 of RUN_ANALYSIS | 13 features exactly as in DATA_MODEL.md section 8 | Feature logic is SQL anyone can read and audit |
| 11 | PREFERRING (skyline queries) |
step 11 of RUN_ANALYSIS | Peels Pareto layers of companies that are best on return, risk and amount together; the top 50 go to RMs | Multi-criteria ranking with no hand-tuned weights |
| 12 | Roles and users | 07_security.sql | One Exasol user per app role. Loan operations cannot read the raw API data; RMs see a masked view. | Access is enforced by the database, not only by the app |
| 13 | Purge with proof | 08_purge.sql | Raw CIBIL, GST and AA replies are deleted at their purge time, with row counts and a SHA-256 fingerprint kept | Consent-based data (Account Aggregator) must not be kept |
| 14 | Generative AI from a UDF | 09_ai_udf.sql | CREDIT_MEMO asks Claude for a short credit memo per company, on demand from the Exasol report page; memos are stored in CREDORA.MEMOS (with Bedrock instead of the Anthropic API, the backend calls the model) |
LLM calls are just another SQL function over governed data |
| 15 | Exasol MCP server | backend/ | "Ask Credora": plain-English questions answered by Claude through the official Exasol MCP server (exasol-mcp-server 2.3.0, started by the backend over stdio), as read-only CREDORA_MCP; the UI shows every SQL query it ran |
AI that can only read what its role allows |
| 16 | Exasol Personal launcher on AWS | credora.py | One command installs Exasol on EC2, with its ports limited to the deployer's IP | Reproducible from the repo by anyone with an AWS account |
| 17 | SQL preprocessor, multiplexed (Lua) | 10_semantic_views_module.sql, exasol_setup.py activate_preprocessor |
Exasol has one sql_preprocessor_script slot. The preprocessor framework's dispatcher PREPROC_RT.MASTER owns it system-wide and runs, per user, the modules their roles' rules name. The semantic layer is one module, PREPROC_RT.SEMANTIC_VIEWS_V1, scoped to R_MCP, R_ADMIN, R_DATASCI, R_LOANOPS and R_RM; CREDORA_SVC has no rule, so its sessions drop the preprocessor after their first statement |
More SQL extensions can be added later as rules, without a new owner for the slot. A failing module is skipped, never blocking a statement |
| 18 | Semantic layer with certified metrics | exasol/semantic/ | CREDORA_SEM.PORTFOLIO: one row per company of the latest completed run, with 15 certified metrics (approved companies, approval rate, pre-approved amount, negotiation headroom, average offer ROI, average PD, expected loss, top 50 amount and more), synonyms, agent instructions and verified example questions. Ask Credora answers from it first |
One definition of each number for the web app, analysts and the AI; the model refuses a metric and dimension pair that would double count |
An ecosystem of building blocks#
Exasol is more than a database engine. Around it, the exasol and exasol-labs GitHub organisations publish open-source building blocks: virtual schema adapters, script language containers, drivers, an MCP server, a preprocessor framework, a semantic layer and the Exasol Personal launcher. A custom solution picks the blocks it needs and plugs them together, like Lego, instead of building connectors and runtimes itself. Credora is assembled this way:
| Building block | Repository | Used in Credora for |
|---|---|---|
Exasol Personal launcher (exasol CLI, AWS and Ubuntu presets) |
exasol/exasol-personal | Installing Exasol on EC2 with one command, from the deploy tool |
| Oracle virtual schema | exasol/oracle-virtual-schema | VS_CBS: core banking read in place |
| Virtual schema framework and dialects | exasol/virtual-schemas | The adapter model both virtual schemas follow |
| ADBC virtual schema (Rust) | exasol-labs/adbc-vs-rs | VS_CIF on MySQL and VS_LOS on PostgreSQL, one adapter for both |
| Rust script language container | exasol-labs/language-container-rs | The runtime the ADBC adapter runs in, pinned to 0.23.1 |
| Built-in Python 3 and Java script languages | exasol/script-languages-release | API, ML and AI UDFs (Python); the Oracle adapter (Java) |
| pyexasol | exasol/pyexasol | The backend's and the deploy tool's connection to Exasol |
| Exasol MCP server | exasol/mcp-server | Ask Credora: Claude queries Exasol through MCP as a read-only user |
| Preprocessor framework | exasol-labs/preprocessor-framework | Owns the one system SQL preprocessor slot and dispatches per role; pinned to v0.9.1 |
| Semantic views | exasol-labs/exasol-semantic-views | The semantic model CREDORA_SEM.PORTFOLIO and its compiler, plugged into the framework as a module; pinned to v0.5.1 |
| Personal local starter kit | exasol-labs/exasol-personal-local-starterkit | Local prototyping before the cloud build: MCP setup, exapump, pyexasol |
Each block is independent and versioned, so one can be upgraded or swapped without touching the others. The deploy tool pins the version of every block it downloads. Credora's own code adds only the bank-specific parts: the data model, the API and ML UDFs, the decision logic in Lua, the web app and the infrastructure.
Measured on the live deployment#
Step of RUN_ANALYSIS |
Rows | Time |
|---|---|---|
| Refresh 3 virtual schemas | 3 | 1.7 s |
| Read applications (PostgreSQL joined with MySQL) | 1,000 | 0.3 s |
| CIBIL consumer reports (API) | 3,490 | 0.7 s |
| CIBIL commercial reports (API) | 1,000 | 0.4 s |
| GST returns (API) | 71,217 | 1.4 s |
| Account Aggregator (API) | 5,810 | 3.2 s |
| Features (Oracle pushdown, 850k transactions) | 1,000 | 10.4 s |
| Score (ML UDF) | 1,000 | 1.9 s |
| Decide offer and negotiation limit | 1,000 | 0.06 s |
| Rank with PREFERRING | 83 | 0.02 s |
| Total | about 20 s |
SQL preprocessing and the semantic layer#
How the two blocks fit together:
deploy/credora.py exasolinstalls preprocessor-framework v0.9.1 (release.sql, sha256-pinned) and exasol-semantic-views v0.5.1 (the release's own installer, from its prebuiltbuild/install/), then the module, the latest-run views and the model, and last setsALTER SYSTEM SET SQL_PREPROCESSOR_SCRIPT = PREPROC_RT.MASTERafter trying it in one session.- The semantic views project ships its own preprocessor,
SEMANTIC_ADMIN.SEMANTIC_PREPROCESSOR, written to own the slot; the framework lists the integration as a follow-on and ships no module for it. Credora follows the framework's "bring your own preprocessor" contract:PREPROC_RT.SEMANTIC_VIEWS_V1is a function that takes the statement and returns the semantic compiler's SQL, or nothing. It is registered withPREPROC.MODULE_REGISTER(self-deployed, fronting exasol-semantic-views v0.5.1) and onePREPROC.RULE_ENSUREper role. A semantic query the layer refuses fails with the layer's own message (for exampleSEMANTIC_QUERY_020: Unknown semantic field). - Cost for a role user's ordinary statement: about 4 ms (a probe of
<schema>.SEMANTIC_DISCOVERY); the compiler is loaded only for statements that readCREDORA_SEM. - Ask Credora needs no preprocessor tools: every MCP database session, including the new one the MCP server opens after a failed query, gets semantic SQL from the system setting. The tools are switched off (
enable_preprocessor_tools: false).
Checks:
SELECT * FROM PREPROC_RT.MY_PIPELINE; -- what runs on my statements (no rows: nothing)
SELECT * FROM PREPROC_RT.INSTALLED_MODULES; -- semantic_views, v0.5.1
EXECUTE SCRIPT PREPROC.STATUS; -- resolution CURRENT, as a DBA
SELECT industry, approved_companies, pre_approved_amount
FROM CREDORA_SEM.PORTFOLIO GROUP BY industry ORDER BY pre_approved_amount DESC LIMIT 5;
Roll back: ALTER SYSTEM SET SQL_PREPROCESSOR_SCRIPT = NULL; (new sessions; the app's pooled sessions keep their setting until they reconnect). Semantic SQL then needs EXECUTE SCRIPT SEMANTIC_ADMIN.ENABLE_SEMANTIC_SQL(); per session.
After role grants change, run EXECUTE SCRIPT PREPROC.REFRESH; (the deploy tool does).
Upgrading:
- exasol-semantic-views: change
SEMVIEWS_VERSIONand its sha256 indeploy/exasol_setup.pyand re-runcredora.py exasol. The installer runs when the pin differs fromSEMANTIC_CATALOG.PRODUCT_VERSION, with the system preprocessor off while it replaces the scripts, as the project's upgrade procedure asks. - preprocessor-framework:
release.sqlis a clean-install artifact, so the deploy tool runs it only when schemaPREPROCdoes not exist, and records the release as that schema's comment. For a new release, run the framework's idempotent installer (uv run preproc installfrom its checkout at the new tag, as DBA), which keepsPREPROC.CONFIG, then updatePREPROC_VERSION, its sha256 and the schema comment.