Why Exasol, feature by feature

Credora uses Exasol as the place where a bank's scattered data meets, is analysed and becomes a decision. This page lists every Exasol feature the project uses, where it is in the code, and why it is the right tool for that job.

# Exasol feature Where What it does in Credora Why it matters
1 Virtual schema: Oracle (Java adapter, JDBC) 03_virtual_schemas.sql VS_CBS reads 850,000 core-banking transactions in Oracle in place Core banking is never copied. Banks keep their system of record where it is.
2 Virtual schema: adbc-vs-rs (Rust, ADBC, Arrow) same VS_CIF on MySQL and VS_LOS on PostgreSQL One adapter for two engines, moving Arrow record batches, not rows
3 Custom script language container exasol_setup.py install_rust_slc Rust SLC 0.23.1 uploaded to BucketFS and registered as RUST Runs the Rust adapter at the SDK version it was built for
4 BucketFS exasol_setup.py, 05_ml_udfs.sql Holds the SLC, adapter library, ADBC drivers (fetched on first use), the Oracle JDBC driver, and every trained model version One distributed file store for code, drivers and models
5 Python UDFs calling APIs (SET scripts) 04_api_udfs.sql CIBIL, GSTN and Account Aggregator data fetched by 16 parallel UDF instances through bulk endpoints: 3,490 CIBIL reports in under 1 second External data comes in with SQL, in parallel, and lands next to the bank data
6 CONNECTION objects 02_connections.sql API addresses, tokens and database credentials Secrets never appear in SQL text, and access is granted per role
7 ML training inside the database TRAIN_PD in 05_ml_udfs.sql Gradient-boosted trees with monotonic credit constraints, trained on 5,000 past loans read live from PostgreSQL, measured on a hold-out set (AUC, Gini, permutation importance) No data export to a separate ML platform
8 ML scoring with explanations PD_SCORE Probability of default for 1,000 companies in about 2 seconds, plus the effect of each feature on each company Every decision can say why
9 Lua scripting 06_run_analysis.sql RUN_ANALYSIS runs 11 logged steps; TRAIN_MODEL versions models The whole pipeline is one database call, with a timing for each step
10 Analytic SQL: window functions, CTEs, pushdown aggregation step 7 of RUN_ANALYSIS 13 features exactly as in DATA_MODEL.md section 8 Feature logic is SQL anyone can read and audit
11 PREFERRING (skyline queries) step 11 of RUN_ANALYSIS Peels Pareto layers of companies that are best on return, risk and amount together; the top 50 go to RMs Multi-criteria ranking with no hand-tuned weights
12 Roles and users 07_security.sql One Exasol user per app role. Loan operations cannot read the raw API data; RMs see a masked view. Access is enforced by the database, not only by the app
13 Purge with proof 08_purge.sql Raw CIBIL, GST and AA replies are deleted at their purge time, with row counts and a SHA-256 fingerprint kept Consent-based data (Account Aggregator) must not be kept
14 Generative AI from a UDF 09_ai_udf.sql CREDIT_MEMO asks Claude for a short credit memo per company, on demand from the Exasol report page; memos are stored in CREDORA.MEMOS (with Bedrock instead of the Anthropic API, the backend calls the model) LLM calls are just another SQL function over governed data
15 Exasol MCP server backend/ "Ask Credora": plain-English questions answered by Claude through the official Exasol MCP server (exasol-mcp-server 2.3.0, started by the backend over stdio), as read-only CREDORA_MCP; the UI shows every SQL query it ran AI that can only read what its role allows
16 Exasol Personal launcher on AWS credora.py One command installs Exasol on EC2, with its ports limited to the deployer's IP Reproducible from the repo by anyone with an AWS account
17 SQL preprocessor, multiplexed (Lua) 10_semantic_views_module.sql, exasol_setup.py activate_preprocessor Exasol has one sql_preprocessor_script slot. The preprocessor framework's dispatcher PREPROC_RT.MASTER owns it system-wide and runs, per user, the modules their roles' rules name. The semantic layer is one module, PREPROC_RT.SEMANTIC_VIEWS_V1, scoped to R_MCP, R_ADMIN, R_DATASCI, R_LOANOPS and R_RM; CREDORA_SVC has no rule, so its sessions drop the preprocessor after their first statement More SQL extensions can be added later as rules, without a new owner for the slot. A failing module is skipped, never blocking a statement
18 Semantic layer with certified metrics exasol/semantic/ CREDORA_SEM.PORTFOLIO: one row per company of the latest completed run, with 15 certified metrics (approved companies, approval rate, pre-approved amount, negotiation headroom, average offer ROI, average PD, expected loss, top 50 amount and more), synonyms, agent instructions and verified example questions. Ask Credora answers from it first One definition of each number for the web app, analysts and the AI; the model refuses a metric and dimension pair that would double count

An ecosystem of building blocks#

Exasol is more than a database engine. Around it, the exasol and exasol-labs GitHub organisations publish open-source building blocks: virtual schema adapters, script language containers, drivers, an MCP server, a preprocessor framework, a semantic layer and the Exasol Personal launcher. A custom solution picks the blocks it needs and plugs them together, like Lego, instead of building connectors and runtimes itself. Credora is assembled this way:

Building block Repository Used in Credora for
Exasol Personal launcher (exasol CLI, AWS and Ubuntu presets) exasol/exasol-personal Installing Exasol on EC2 with one command, from the deploy tool
Oracle virtual schema exasol/oracle-virtual-schema VS_CBS: core banking read in place
Virtual schema framework and dialects exasol/virtual-schemas The adapter model both virtual schemas follow
ADBC virtual schema (Rust) exasol-labs/adbc-vs-rs VS_CIF on MySQL and VS_LOS on PostgreSQL, one adapter for both
Rust script language container exasol-labs/language-container-rs The runtime the ADBC adapter runs in, pinned to 0.23.1
Built-in Python 3 and Java script languages exasol/script-languages-release API, ML and AI UDFs (Python); the Oracle adapter (Java)
pyexasol exasol/pyexasol The backend's and the deploy tool's connection to Exasol
Exasol MCP server exasol/mcp-server Ask Credora: Claude queries Exasol through MCP as a read-only user
Preprocessor framework exasol-labs/preprocessor-framework Owns the one system SQL preprocessor slot and dispatches per role; pinned to v0.9.1
Semantic views exasol-labs/exasol-semantic-views The semantic model CREDORA_SEM.PORTFOLIO and its compiler, plugged into the framework as a module; pinned to v0.5.1
Personal local starter kit exasol-labs/exasol-personal-local-starterkit Local prototyping before the cloud build: MCP setup, exapump, pyexasol

Each block is independent and versioned, so one can be upgraded or swapped without touching the others. The deploy tool pins the version of every block it downloads. Credora's own code adds only the bank-specific parts: the data model, the API and ML UDFs, the decision logic in Lua, the web app and the infrastructure.

Measured on the live deployment#

Step of RUN_ANALYSIS Rows Time
Refresh 3 virtual schemas 3 1.7 s
Read applications (PostgreSQL joined with MySQL) 1,000 0.3 s
CIBIL consumer reports (API) 3,490 0.7 s
CIBIL commercial reports (API) 1,000 0.4 s
GST returns (API) 71,217 1.4 s
Account Aggregator (API) 5,810 3.2 s
Features (Oracle pushdown, 850k transactions) 1,000 10.4 s
Score (ML UDF) 1,000 1.9 s
Decide offer and negotiation limit 1,000 0.06 s
Rank with PREFERRING 83 0.02 s
Total about 20 s

SQL preprocessing and the semantic layer#

How the two blocks fit together:

  • deploy/credora.py exasol installs preprocessor-framework v0.9.1 (release.sql, sha256-pinned) and exasol-semantic-views v0.5.1 (the release's own installer, from its prebuilt build/install/), then the module, the latest-run views and the model, and last sets ALTER SYSTEM SET SQL_PREPROCESSOR_SCRIPT = PREPROC_RT.MASTER after trying it in one session.
  • The semantic views project ships its own preprocessor, SEMANTIC_ADMIN.SEMANTIC_PREPROCESSOR, written to own the slot; the framework lists the integration as a follow-on and ships no module for it. Credora follows the framework's "bring your own preprocessor" contract: PREPROC_RT.SEMANTIC_VIEWS_V1 is a function that takes the statement and returns the semantic compiler's SQL, or nothing. It is registered with PREPROC.MODULE_REGISTER (self-deployed, fronting exasol-semantic-views v0.5.1) and one PREPROC.RULE_ENSURE per role. A semantic query the layer refuses fails with the layer's own message (for example SEMANTIC_QUERY_020: Unknown semantic field).
  • Cost for a role user's ordinary statement: about 4 ms (a probe of <schema>.SEMANTIC_DISCOVERY); the compiler is loaded only for statements that read CREDORA_SEM.
  • Ask Credora needs no preprocessor tools: every MCP database session, including the new one the MCP server opens after a failed query, gets semantic SQL from the system setting. The tools are switched off (enable_preprocessor_tools: false).

Checks:

sql
SELECT * FROM PREPROC_RT.MY_PIPELINE;            -- what runs on my statements (no rows: nothing)
SELECT * FROM PREPROC_RT.INSTALLED_MODULES;      -- semantic_views, v0.5.1
EXECUTE SCRIPT PREPROC.STATUS;                   -- resolution CURRENT, as a DBA
SELECT industry, approved_companies, pre_approved_amount
FROM CREDORA_SEM.PORTFOLIO GROUP BY industry ORDER BY pre_approved_amount DESC LIMIT 5;

Roll back: ALTER SYSTEM SET SQL_PREPROCESSOR_SCRIPT = NULL; (new sessions; the app's pooled sessions keep their setting until they reconnect). Semantic SQL then needs EXECUTE SCRIPT SEMANTIC_ADMIN.ENABLE_SEMANTIC_SQL(); per session.

After role grants change, run EXECUTE SCRIPT PREPROC.REFRESH; (the deploy tool does).

Upgrading:

  • exasol-semantic-views: change SEMVIEWS_VERSION and its sha256 in deploy/exasol_setup.py and re-run credora.py exasol. The installer runs when the pin differs from SEMANTIC_CATALOG.PRODUCT_VERSION, with the system preprocessor off while it replaces the scripts, as the project's upgrade procedure asks.
  • preprocessor-framework: release.sql is a clean-install artifact, so the deploy tool runs it only when schema PREPROC does not exist, and records the release as that schema's comment. For a new release, run the framework's idempotent installer (uv run preproc install from its checkout at the new tag, as DBA), which keeps PREPROC.CONFIG, then update PREPROC_VERSION, its sha256 and the schema comment.