video-iac/k8s/monitoring/README.md

# Monitoring and Alerting

## Slack Webhook Configuration

The Slack notifications use a specific Incoming Webhook URL structure:
`https://hooks.slack.com/services/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`

These ID components represent:
- **T0L5RA3E0**: Slack Workspace ID (e.g., JamKazam)
- **B01SM8RC346**: Bot/App Configuration ID (unique to the specific "Incoming Webhook" integration created in the Slack app management)
- **XDDOrcPE7eAXJPMCvc5FxIva**: The Secret Token for authentication

### Updating the Webhook
If you need to change the channel or regenerate the URL:
1.  Go to [Slack App Management](https://api.slack.com/apps).
2.  Select the relevant App (e.g., "Monitoring" or "Incoming Webhooks").
3.  Navigate to **Incoming Webhooks**.
### Generating New Webhooks for Specific Channels

If you need to send alerts to a **new channel** (like `#video-cluster-prd-alerts`), the existing URL might not work if it's locked to a single channel. To create a new one:

1.  Navigate to [Slack App Management](https://api.slack.com/apps).
2.  Select your App (likely named "Prometheus Alerts", "Incoming Webhooks", or similar).
3.  In the left sidebar, click **Incoming Webhooks**.
4.  Scroll to the bottom and click the button **"Add New Webhook to Workspace"**.
5.  Select the **specific channel** you want to post to (e.g., `#video-cluster-prd-alerts`).
6.  Click **Allow**.
7.  Copy the **new Webhook URL** generated in the table.
8.  Update the `api_url` in the corresponding `values.yaml` file.