Attribute Lifecycle

How attributes are created, versioned, and managed over time, including schema evolution and version activation/deactivation.

Discovered Attributes

Creation process:

1. Created during scan via sp_upsert_attribute

2. Attribute record in core.datasource_attribute

3. All 7 polymorphic versions created in core.datasource_attribute_polymorphic_version

4. Matching versions set to VERSION_STATUS = 'active'; others remain 'inactive'

5. VERSION_STATUS_DATE set for activated versions

---

Schema Attributes

Similar to Discovered, but created from DATASOURCE_OBJECT_SCHEMA_SAMPLE when ATTRIBUTE_CREATE_TYPE = 'Schema'. All 7 polymorphic versions created; matching versions activated. Updated when full scans are performed.

---

Virtual Attributes

Created via UI or sp_upsert_virtual_datasource_attribute. Single polymorphic version (no polymorphism). Automatically set to INCLUDE_IN_CODE_GEN = TRUE.

---

Version Management

Active (VERSION_STATUS = 'active'):

• Included in code generation (if INCLUDE_IN_CODE_GEN = TRUE)

• Data type found in source data

• VERSION_STATUS_DATE tracks activation time

Inactive (VERSION_STATUS = 'inactive'):

• Not included in code generation

• Ready for activation if data type appears

• Created proactively when attribute is first discovered

Status transitions:

• Inactive → Active: When data type discovered in scans

• Active → Inactive: When data type disappears (schema evolution)

---

Version Status Date

VERSION_STATUS_DATE tracks when a version was created or last activated. Used for schema drift detection and version history.

---

Schema Evolution

How it works:

1. Existing inactive versions activated when their data type appears (no new records needed)

2. Active versions may become inactive if data type disappears

3. VERSION_STATUS_DATE updated on activation/deactivation

4. Users review and configure newly activated versions

Example:

• Initial: status discovered as str → status_str active, others inactive

• Schema change: status appears as int → status_int activated (existing version)

• Result: Both status_str and status_int active; no new records created

---

Attribute Updates

Update methods:

• UI - Data Source Attributes screen

• Bulk CSV - update_attribute_metadata_core stored procedure

• API - sp_update_attribute_polymorphic_versions stored procedure

Editable fields: See DataPancake Metadata Fields for complete reference. Key categories:

• Snowflake data type configuration (precision, scale, datetime formats)

• Transformation expressions and null handling

• Column naming (aliases)

• Code generation flags (INCLUDE_IN_CODE_GEN, PARENT_INCLUDE_IN_CODE_GEN)

• Semantic layer configuration

• Security policies (row-level access, masking)

---

Lifecycle Management

Monitoring schema changes:

• Review VERSION_STATUS_DATE to identify recent activations

• Monitor vw_datasource_version_all_attributes for schema drift

• Review newly activated versions after scans

Version management:

• Set INCLUDE_IN_CODE_GEN = FALSE for versions you don't need

• Review inactive versions periodically (they're ready for activation)

• Configure newly activated versions after schema evolution