How to Create a Single DataPancake Data Source (Script Builder)

Build a script to create and initiate a scan for a single data source in DataPancake and optionally generate SQL code.

Please ensure the Admin Guides have been completed before creating your first data source.

This script builder is not available for the following scenarios:

• Data sources with a pre-configured schema

• Data sources that contain stream data such as Kafka

See "How to Create a DataPancake Data Source (UI)" if needed.

1. Navigate to the "Data Source" script builder

2. On the "Single Data Source" tab, enter the custom data source name

3. (Optional) Change the app database name

The default is DATAPANCAKE

4. Enter the role used to install DataPancake

5. Enter the role which has the select privilege for the database object to be scanned

6. Select the Snowflake Database Object Type

7. Select the warehouse to be used by DataPancake to scan data

8. Enter the Semi-structured Data Format Type

9. Enter the Snowflake database name that contains the data

If the name is case sensitive, put the name in double quotes such as "EXAMPLE_NAME"

10. (Optional) Indicate if the database is shared

11. Enter the name of the schema which contains the database object to be scanned

If the name is case sensitive, put the name in double quotes such as "EXAMPLE_NAME"

12. Enter the name of the database object to be scanned

If the name is case sensitive, put the name in double quotes such as "EXAMPLE_NAME"

13. Enter the name of the column to be scanned

If the name is case sensitive, put the name in double quotes such as "EXAMPLE_NAME"

14. (Optional) Prevent the scan from starting automatically

If you switch this off, you will need to manually trigger the data source scan from the "Scan Data" page.

15. (Optional) Enter a record limit for the initial scan

If you want to scan all records, set the limit to 0

The "Generate Code After Scan" will automatically create the output SQL statements based on the discovered schema at the completion of the scan.

SQL Code Generation is a paid feature available once you purchase the application or as part of an Enterprise license. Code Generation in the Trial version is not enabled.

If you choose not to generate code, skip to step 21.

16. Enable the toggle to add SQL code generation features after the scan is complete.

17. Enter the user defined name for the root dynamic table, such as 'dt_<object_name>'

18. Enter the name of the warehouse used to refresh dynamic tables

This warehouse name is included as part of the create dynamic SQL table statements.

"Why is this not a dropdown with options like the scan data warehouse one above?"

The previous dropdown is the warehouse used to scan data, which must be connected to DataPancake.

The warehouse here is used to refresh dynamic tables and can be any warehouse, including ones not connected to DataPancake.

19. (Optional) Enter the type of semantic layer view to create

20. Enter the user defined name for the root level view, such as 'vw_<object_name>'

21. Click "Generate Script"

22. Copy the script

23. Open a new snowflake worksheet and paste the script

24. Review & Execute the script

Use the blue dropdown arrow at the top right and click "Run All"

25. Ensure successful execution by examining the call result column in the result set

If you mistyped any of the database connection information, the resulting error message will show you what was incorrect.

If the warehouse used has not been previously added to DataPancake, then the scan will not be initiated.

If the scan is initiated, the result message will show that the scan has started.