Schema Summary Views

Views for data source overview, scan history, configurations, and billing information.

VW_DATASOURCES

Comprehensive summary of all data sources with configuration and metadata.

Contents: Data source identification, source connection details, data format/type, scan statistics, product tier flags, output configuration, metadata table configuration, Gen AI instructions

Use cases: Data source inventory, configuration review, product tier verification, integration workflows

Example Query:

Example

SELECT 
    datasource_name,
    data_format,
    last_scan,
    attribute_count,
    polymorphic_attribute_count,
    product_tier_attribute_metadata_management,
    product_tier_sql_code_generation
FROM core.vw_datasources
WHERE datasource_status = 'active'
ORDER BY datasource_name;

---

VW_DATASOURCE_SCANS

Historical scan execution records with performance metrics.

Contents: Scan identification, timing (begin, end, duration), status, record counts, performance metrics (records per minute), configuration details (WHERE clause, ORDER BY), warehouse information, thread/batch details

Use cases: Scan history review, performance analysis, troubleshooting, monitoring completion

Example Query:

Example

SELECT 
    datasource_name,
    configuration_name,
    scan_datetime_begin,
    scan_duration_minutes,
    records_processed,
    records_processed_per_minute,
    scan_status
FROM core.vw_datasource_scans
WHERE datasource_id = 'your-datasource-id'
ORDER BY scan_datetime_begin DESC;

---

VW_DATASOURCE_SCANS_IN_PROCESS

Real-time monitoring of currently executing scans with progress estimates.

Contents: Active scan identification, progress tracking (records processed, estimated remaining), time estimates (duration, time remaining, percentage complete), performance metrics, thread configuration, batch status, warehouse information

Use cases: Monitoring active scans, progress tracking, performance optimization, resource utilization

Example Query:

Example

SELECT 
    datasource_name,
    configuration_name,
    scan_status,
    percentage_complete,
    estimated_records_processed,
    estimated_records_remaining,
    process_time_remaining_in_minutes
FROM core.vw_datasource_scans_in_process
ORDER BY scan_datetime_begin DESC;

---

VW_DATASOURCE_SCAN_CONFIGURATIONS

All scan configuration settings for data sources.

Contents: Configuration identification, record limits/filtering (SCAN_RECORD_LIMIT, SCAN_WHERE_CLAUSE, SCAN_ORDER_BY), scheduling (MONITOR_ENABLED, MONITOR_CRON_SCHEDULE, MONITOR_CRON_TIMEZONE), vertical scale settings (PROCEDURE_INSTANCE_COUNT, PROCEDURE_INSTANCE_ROW_COUNT, thread count), warehouse assignment, CODE_GENERATE_ON_VERSION_CHANGE, ATTRIBUTE_CREATE_TYPE

Use cases: Configuration review, comparison, integration workflows, audit

Example Query:

Example

SELECT 
    datasource_name,
    configuration_name,
    enable_schedule,
    cron_schedule,
    number_of_procedure_calls,
    thread_count,
    warehouse_name
FROM core.vw_datasource_scan_configurations
WHERE datasource_id = 'your-datasource-id';

---

VW_DATASOURCE_SCAN_ERRORS

Error records from scan operations for diagnostics and troubleshooting.

Contents: Error identification, scan/batch information, error messages, timestamps, status

Use cases: Error diagnostics, troubleshooting scan failures, quality monitoring

Example Query:

Example

SELECT 
    datasource_name,
    configuration_name,
    error_message,
    scan_datetime_begin
FROM core.vw_datasource_scan_errors
WHERE datasource_id = 'your-datasource-id'
ORDER BY scan_datetime_begin DESC
LIMIT 100;

---

VW_DATASOURCE_SCAN_PROCEDURE_CALLS

Detailed procedure call information for scans using multiple procedure calls.

Contents: Procedure call identification (procedure number, scan ID), timing (start, end, duration), records processed per call, performance metrics (records per minute per procedure), thread/batch information, version information

Use cases: Performance analysis for multi-call scans, procedure call optimization, timeout management

Example Query:

Example

SELECT 
    datasource_name,
    procedure_number,
    procedure_records_processed,
    procedure_records_processed_per_minute,
    procedure_start_time,
    procedure_end_time
FROM core.vw_datasource_scan_procedure_calls
WHERE datasource_scan_id = 'your-scan-id'
ORDER BY procedure_number;

---

VW_DATASOURCE_SCAN_COMPUTE_RESOURCE_USE

Active scan compute resource utilization for warehouse management.

Contents: Scan/data source identification, warehouse information, scan status/progress, record counts

Use cases: Warehouse utilization monitoring, resource conflict detection, capacity planning

Note: This view is not explicitly defined in the codebase; may be referenced but not created.

Example Query:

Example

SELECT 
    compute_resource_name,
    datasource_name,
    scan_status,
    records_processed
FROM core.vw_datasource_scan_compute_resource_use
WHERE scan_status != 'Completed' AND scan_status != 'Failed';

---

VW_DATASOURCE_BILLING_EVENTS

Billing event history for product tier upgrades and scan operations.

Contents: Event identification (type, description, timestamp), data source information (current and historical names), product tier information, billing details (month, year, pricing tier, price, charge), attribute quantity, version information, scan association

Note: Created when product tier is upgraded for a data source or a scan is performed in a given month.

Use cases: Billing review/audit, cost analysis, product tier history, financial reporting

Example Query:

Example

SELECT 
    current_datasource_name,
    product_tier_name,
    event_type,
    billing_month,
    billing_year,
    price,
    charge
FROM core.vw_datasource_billing_events
WHERE billing_year = 2024
ORDER BY billing_month DESC, created_datetime DESC;

---

VW_DATASOURCE_PRODUCT_TIER_AUDIT_LOG

Historical record of product tier changes for data sources.

Contents: Product tier change history, timestamps, user information, data source information (current and historical), source connection details at time of change, record status

Note: Created when product tier changes for a data source.

Use cases: Product tier change audit trail, historical configuration review, compliance/governance, change tracking

Example Query:

Example

SELECT 
    current_datasource_name,
    product_tier_name,
    date_added,
    added_by_user,
    datasource_name_at_time_of_change
FROM core.vw_datasource_product_tier_audit_log
WHERE datasource_id = 'your-datasource-id'
ORDER BY date_added DESC;