Attribute Lifecycle

How attributes are created, versioned, and managed over time, including schema evolution and version activation/deactivation.

Discovered Attributes

1

Created during scan process.

2

Attribute record created in datasource_attribute table.

3

All 7 polymorphic versions proactively created in datasource_attribute_polymorphic_version table.

4

Only the version(s) matching the discovered data type are activated (status = 'active').

5

Non-matching versions are created but remain inactive (status = 'inactive').

6

Initial status dates set for activated versions.

Key Point: All 7 versions are created immediately, not only when polymorphism is detected. This proactive approach ensures the system is always ready for schema changes.

---

Schema Attributes

• Similar to Discovered attributes, but based on schema sample.

• All 7 polymorphic versions are created upfront.

• Versions matching the schema sample are activated.

• Can be updated when full scans are performed.

---

Virtual Attributes

1

Created via UI or API (sp_upsert_virtual_datasource_attribute).

2

User specifies name, types, and transformation expression.

3

Single polymorphic version created (no polymorphism for virtual attributes); automatically set to INCLUDE_IN_CODE_GEN = TRUE.

Note: Virtual attributes don't have multiple polymorphic versions because they're user-defined and don't come from source data.

---

Version Management

Active vs. Inactive

• Active Versions:

  • Included in code generation

  • Represent data types that have been found in the data

  • Have status date set to when they were activated

• Inactive Versions:

  • Preserved for history but not used in code generation

  • Ready to be activated if that data type appears

  • Created proactively when attribute is first discovered

Status Transitions:

• Versions become inactive when schema changes or attributes are deprecated.

• Inactive versions become active when their data type is discovered.

• Status dates track when versions were activated/deactivated.

---

Version Status Date

Purpose:

• Tracks when each version was created or last activated

• Used for schema drift detection and version history

• Helps identify when schema changes occurred

Usage:

• Monitor version status dates to detect schema evolution

• Review changes in version activation dates

• Track attribute lifecycle over time

---

Schema Evolution

How Schema Evolution Works

1

Existing inactive versions are activated if their data type appears (no new versions needed — they already exist).

2

Active versions may become inactive if that data type disappears from the data.

3

Version status dates updated to track when versions were activated/deactivated.

4

Users can review changes and configure newly activated versions.

Key Benefit: Because all 7 versions are created upfront, schema evolution doesn't require creating new polymorphic version records—only activating existing ones. This makes schema drift handling seamless and efficient.

Example: Schema Evolution

Initial State:

• Attribute status discovered as string

• status_string → active

• All other 6 versions → inactive

After Schema Change:

• New record contains status as number

• status_int → activated (was already created!)

• status_string → remains active

• Both versions now active

Result: No new records needed, just activation of existing versions.

---

Attribute Updates

Update Methods

Attributes can be updated through:

• UI — Data Source Attributes screen

• Bulk CSV operations — Update multiple attributes via CSV upload

• API — Stored procedures for programmatic updates

Updateable Fields

• Type Configuration:

  • Data types, precision, scale

  • DateTime formats

  • Use DateTime Format flag

• Transformation:

  • Transformation expressions

  • Null value expressions

  • Transformation type

• Naming:

  • Alias names

  • Semantic layer alias names

• Code Generation:

  • Include in code gen flags

  • Parent include in code gen flags

• Semantic Layer:

  • Semantic layer configurations

  • Semantic layer expressions

• Security:

  • Security policy settings

  • Masking policy configuration

And many more metadata fields

---

Best Practices for Lifecycle Management

Monitoring Schema Changes

1

Review Version Status Dates

• Check when versions were activated

• Identify recent schema changes

• Monitor for unexpected activations

2

Track Inactive Versions

• Understand which versions are ready for activation

• Review periodically for potential future use

• Document why certain versions remain inactive

3

Handle Schema Evolution

• Review newly activated versions after scans

• Configure newly activated versions appropriately

• Update transformation expressions if needed

Version Management

1

Keep Active Only What You Need

• Deactivate versions that aren't needed

• Reduces complexity in code generation

• Improves performance

2

Document Version Decisions

• Note why certain versions are active/inactive

• Document transformation logic for each version

• Maintain version history

3

Regular Reviews

• Periodically review all versions

• Check for unexpected activations

• Clean up unused inactive versions if needed