Manage Scan Configuration
Last updated
Last updated
Begin by selecting the data source you wish to create a Scan Configuration for from the dropdown at the top of the page.
You will see a table populated by all the available Scan Configurations for the currently selected data source. This table includes details about the Scan Configuration such as a record limit, if the config has an order by or where clause, how many procedure calls the config is broken into, and if the config has a monitoring schedule in place.
Data sources added to Pancake via worksheet in a quick start will often have an existing quick Scan Configuration with a record limit of 150. Otherwise, this table will be blank until a Scan Config is created.
You can set the name of the scan configuration here, typically a user will want to include details about the data source, the size of the scan, or the warehouse - e.g. UDI Full Scan Large.
This is also where you can enable monitoring via a cron schedule. Examples of how cron schedules are rendered are included in a dropdown in this section. Data sources can be scanned at whatever interval the user wishes, but be mindful of how long the scan will last as well as any other scans scheduled on the same warehouse.
In this section, users can set a limit on the number of records Pancake will scan using that config as well as a where clause to limit the scan based on something like a certain attribute or time series.
Leave the record limit at 0 to scan every record in the data source.
Users also select a warehouse from those that have been made available to Pancake. Keep the size and complexity of your data source in mind. We have found that an X-Small is sufficient for a quick scan and that a Medium Snowpark Optimized is optimal for most use cases.
By default, Pancake will scale vertically to utilize all available threads based on the warehouse selected by the user. This is also where you would split up a scan into multiple procedure calls and set an order by.
The details of these settings, how they work, and the conditions under which you would need to alter them are covered as part of the Scan Processing topic.
