Attribute Lifecycle

How attributes are created, versioned, and managed over time, including schema evolution and version activation/deactivation.

Discovered Attributes

Creation process:

1. Created during scan via sp_upsert_attribute
2. Attribute record in core.datasource_attribute
3. All 7 polymorphic versions created in core.datasource_attribute_polymorphic_version
4. Matching versions set to VERSION_STATUS = 'active'; others remain 'inactive'
5. VERSION_STATUS_DATE set for activated versions

---

Schema Attributes

Similar to Discovered, but created from DATASOURCE_OBJECT_SCHEMA_SAMPLE when ATTRIBUTE_CREATE_TYPE = 'Schema'. All 7 polymorphic versions created; matching versions activated. Updated when full scans are performed.

---

Virtual Attributes

Created via UI or sp_upsert_virtual_datasource_attribute. Single polymorphic version (no polymorphism). Automatically set to INCLUDE_IN_CODE_GEN = TRUE.

---

Version Management

Active (VERSION_STATUS = 'active'):

• Included in code generation (if INCLUDE_IN_CODE_GEN = TRUE)
• Data type found in source data
• VERSION_STATUS_DATE tracks activation time

Inactive (VERSION_STATUS = 'inactive'):

• Not included in code generation
• Ready for activation if data type appears
• Created proactively when attribute is first discovered

Status transitions:

• Inactive → Active: When data type discovered in scans
• Active → Inactive: When data type disappears (schema evolution)

---

Version Status Date

VERSION_STATUS_DATE tracks when a version was created or last activated. Used for schema drift detection and version history.

---

Schema Evolution

How it works:

1. Existing inactive versions activated when their data type appears (no new records needed)
2. Active versions may become inactive if data type disappears
3. VERSION_STATUS_DATE updated on activation/deactivation
4. Users review and configure newly activated versions

Example:

• Initial: status discovered as str → status_str active, others inactive
• Schema change: status appears as int → status_int activated (existing version)
• Result: Both status_str and status_int active; no new records created

---

Attribute Updates

Update methods:

• UI - Data Source Attributes screen
• Bulk CSV - update_attribute_metadata_core stored procedure
• API - sp_update_attribute_polymorphic_versions stored procedure

Editable fields: See DataPancake Metadata Fields for complete reference. Key categories:

• Snowflake data type configuration (precision, scale, datetime formats)
• Transformation expressions and null handling
• Column naming (aliases)
• Code generation flags (INCLUDE_IN_CODE_GEN, PARENT_INCLUDE_IN_CODE_GEN)
• Semantic layer configuration
• Security policies (row-level access, masking)

---

Lifecycle Management

Monitoring schema changes:

• Review VERSION_STATUS_DATE to identify recent activations
• Monitor vw_datasource_version_all_attributes for schema drift
• Review newly activated versions after scans

Version management:

• Set INCLUDE_IN_CODE_GEN = FALSE for versions you don't need
• Review inactive versions periodically (they're ready for activation)
• Configure newly activated versions after schema evolution