Notebooks
S
Snowflake Query Cost Monitoring

Query Cost Monitoring

data-sciencenotebookmachine-learningsnowflake-demo-notebooksQuery_Cost_Monitoringdata-engineeringPythonsql
alph-notebooks/snowflake-demo-notebooks / Query_Cost_Monitoring.ipynb

Query Cost Monitoring

A notebook that breaks down compute costs by individual query, allowing teams to identify high-cost operations.

Here's our 4 step process:

  1. SQL query to retrieve query cost data
  2. Convert SQL table to a Pandas DataFrame
  3. Data preparation and filtering (using user input from Streamlit widgets)
  4. Data visualization and exploration

1. Retrieve Data

To gain insights on query costs, we'll write a SQL query to retrieve the credits_used data from the snowflake.account_usage.metering_history table and merging this with associated user, database, schema and warehouse information from the snowflake.account_usage.query_history table.

[ ]

2. Convert Table to a DataFrame

Next, we'll convert the table to a Pandas DataFrame.

[ ]

3. Create an Interactive Slider Widget & Data Preparation

Here, we'll create an interactive slider for dynamically selecting the number of days to analyze. This would then trigger the filtering of the DataFrame to the specified number of days.

Next, we'll reshape the data by calculating the frequency count by hour and task name, which will subsequently be used for creating the heatmap in the next step.

[ ]

4. Create a Heatmap for Visualizing Query Cost

Finally, a heatmap, and stacked bar chart, and bubble chart are generated that will allow us to gain insights on query cost and frequency.

[ ]
[ ]
[ ]

Want to learn more?