HiddenLayer Azure Sentinel Integration Guide

Overview

This guide provides step-by-step instructions for integrating HiddenLayer with Microsoft Sentinel. Runtime detections will be sent from the HiddenLayer Console to an Azure Log Analytics Workspace for ingestion by Sentinel.

Architecture

Prerequisites

Azure subscription with the following permissions:
- Azure Entra ID: Create and manage app registrations, create client secrets
- Azure Monitor: Create and manage Data Collection Endpoints (DCE)
- Azure Monitor: Create and manage Data Collection Rules (DCR)
- Log Analytics Workspace: Create and manage custom tables
- Access Control (IAM): Grant role assignments on Data Collection Rules
Log Analytics Workspace
Azure Sentinel
HiddenLayer Console admin access

Setup Steps

Step 1: Register Application in Azure Entra ID

Navigate to Azure Portal → Microsoft Entra ID → App registrations
Click New registration
Enter a name for the application (e.g., hl-sentinel-log-analytics-stage)
Select Accounts in this organizational directory only (Single tenant)
Click Register
Save the following values:
- Application (client) ID - You’ll need this later
- Directory (tenant) ID - You’ll need this later

Step 2: Create Client Secret

In your app registration, navigate to Certificates & secrets
Click New client secret
Enter a description (e.g., IntegrationToken) and set expiration (recommended: 180 days or 6 months)
Click Add
IMMEDIATELY COPY THE SECRET VALUE - You cannot view it again after leaving this page
Save this value securely - You’ll need it for HiddenLayer configuration
Save Secret ValueThis value will not be available once you leave this page. If you leave the page without copying the secret value, you must create a new secret.

Step 3: Create Data Collection Endpoint

Navigate to Azure Portal → Monitor → Data Collection Endpoints
Click Create
Configure the endpoint:
- Name: hl-sentinel-aidr-<environment> (e.g., hl-sentinel-aidr-stage)
- Subscription: Select your subscription
- Resource Group: Select or create a resource group
- Region: Choose the region closest to your HiddenLayer deployment
Click Review + create → Create
Once created, select the endpoint and click JSON View

Copy the dataCollectionEndpointId - It looks like:

/subscriptions/<subscription-id>/resourceGroups/<rg-name>/providers/Microsoft.Insights/dataCollectionEndpoints/<endpoint-name>

Copy the logsIngestion URL - It looks like:

https://<endpoint-name>-<hash>.<region>.ingest.monitor.azure.com

Step 4: Create Custom Log Analytics Table

Navigate to Azure Portal → Log Analytics workspaces
Select your workspace (where Sentinel is enabled)
Navigate to Settings → Tables
Click Create → New custom log (DCR-based)
Configure the custom log:
- Table name: HiddenLayerAIDRStage (without _CL suffix)
- Data collection endpoint: Select the endpoint created in Step 3
- Data collection rule: Create new or select existing
Click Next
Upload the following JSON schema file:

{
  "TimeGenerated": "2024-10-21T00:01:03.123456Z",
  "conviction_id": "9f891a16-34e6-4e9a-aa5c-22369712e64a",
  "tenant_id": "80ad8fa2-c1f9-430a-a7b9-ad85a9386d45",
  "sensor_id": "8d009b0d-75dc-4287-b0d7-b653c51a5ae1",
  "requester_id": "a_requester_id",
  "source": "aidr",
  "detection_category": "A detection category",
  "attributable_event_id": "8a27bd3e-b7a1-421f-ba43-25f18e595050",
  "concluding_event_id": "2a4c645e-f08e-41a9-882d-8d22cb4b8e41",
  "conviction_timestamp": "2024-10-21T00:01:02.123456Z",
  "mitre": "{ \"Tactic\": { \"UID\": \"AML.TA0001\", \"Name\": \"ML Attack Staging\", \"SrcUrl\": \"https://atlas.mitre.org/tactics/AML.TA001\" }, \"Technique\": { \"UID\": \"AML.T0006\", \"Name\": \"Active Scanning\", \"SrcUrl\": \"https://atlas.mitre.org/tactics/AML.T0006\" } }",
  "severity": "high",
  "engine_name": "fuzzy_correlation"
}

Click Next → Create
Table and Stream NamesAzure will create a table named HiddenLayerAIDRStage_CL and a stream named Custom-HiddenLayerAIDRStage_CL

Step 5: Get Data Collection Rule Details

Navigate to Azure Portal → Monitor → Data Collection Rules
Find and select the data collection rule created in Step 4
Click JSON View
Copy the immutableId - It looks like:
dcr-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Step 6: Grant Permissions to App Registration

Stay on the Data Collection Rule page
Navigate to Access Control (IAM)
Click Add → Add role assignment
Select role: Monitoring Metrics Publisher
Click Next
For Assign access to, select User, group, or service principal
Click Select members
Search for your app registration name (from Step 1)
Select the application
Click Select → Review + assign → Review + assign

Step 7: Configure HiddenLayer Console

Log in to HiddenLayer Console
Navigate to Admin → Integrations
Find Azure Sentinel and click Configure Integration (three dots menu)

Enter the following values:

Field	Value	Notes
Name	`azure-sentinel-integration`	Any descriptive name
Azure Tenant ID	`<tenant-id>`	From Step 1
Azure Client ID	`<client-id>`	From Step 1
Azure Client Secret	`<client-secret-value>`	From Step 2 (the secret value, not the secret ID)
Data Collection Endpoint	`https://<endpoint>.<region>.ingest.monitor.azure.com`	From Step 3 (logsIngestion URL)
Resource ID	`/subscriptions/<sub>/resourceGroups/<rg>/providers/Microsoft.Insights/dataCollectionEndpoints/<name>`	From Step 3 (dataCollectionEndpointId)
Data Collection Rule ID	`dcr-xxxxxxxx...`	From Step 5 (immutableId)
Azure Table Names	`Custom-HiddenLayerAIDRStage_CL`	The stream name with “Custom-” prefix and “_CL” suffix

Click Submit

Configuration Summary

Key Configuration Mapping

What you create in Azure Portal:

Table name: HiddenLayerAIDRStage (no suffix)

What Azure automatically creates:

Log Analytics table: HiddenLayerAIDRStage_CL (adds _CL suffix)
DCR stream: Custom-HiddenLayerAIDRStage_CL (adds Custom- prefix and _CL suffix)

What you configure in HiddenLayer:

Table name: Custom-HiddenLayerAIDRStage_CL (the full stream name)

Critical Field Clarifications

HiddenLayer	Correct Value	Common Mistake
Resource ID	Data Collection Endpoint ID (from DCE)	Using DCR Resource ID instead
Data Collection Rule ID	DCR immutableId (`dcr-xxx` format)	Using full ARM path
Azure Table Names	`Custom-HiddenLayerAIDRStage_CL`	Using `HiddenLayerAIDRStage_CL` or `HiddenLayerAIDRStage`

Verification (Wait 5-10 minutes after configuration)

Step 1: Create an AI Runtime Security detection

In HiddenLayer Console, navigate to LLM Sandbox
Select OWASP Scenarios -> LLM01: Prompt Injection
Turn on Block Prompt Injection in the Policy Settings
Send the prompt
You should see a message saying the prompt was blocked

Step 2: Check Azure for Errors

Run this query in Log Analytics:

DCRLogErrors
| where TimeGenerated > ago(30m)
| where _ResourceId contains "<your-dcr-name>"
| order by TimeGenerated desc

Expected result: No errors If errors exist: See Troubleshooting section below

Step 3: Verify Data Ingestion

Run this query in Log Analytics:

HiddenLayerAIDRStage_CL
| where TimeGenerated > ago(1h)
| order by TimeGenerated desc
| take 10

Expected result: Conviction data appears in the table

Testing the Integration

If you want to manually test the Azure configuration before HiddenLayer sends data, use this Python script:

Test Script

Save as test_integration.py:

#!/usr/bin/env python3
import datetime
import json
from azure.identity import ClientSecretCredential
from azure.monitor.ingestion import LogsIngestionClient

# Configuration - UPDATE THESE VALUES
tenant_id = "YOUR_TENANT_ID"
client_id = "YOUR_CLIENT_ID"
client_secret = "YOUR_CLIENT_SECRET"
dce_endpoint = "https://YOUR_DCE_ENDPOINT.eastus2-1.ingest.monitor.azure.com"
dcr_immutable_id = "dcr-YOUR_IMMUTABLE_ID"
stream_name = "Custom-HiddenLayerAIDRStage_CL"

# Sample conviction data
sample_log = [{
    "TimeGenerated": datetime.datetime.utcnow().isoformat() + "Z",
    "conviction_id": "test-" + datetime.datetime.utcnow().strftime("%Y%m%d%H%M%S"),
    "tenant_id": "test-tenant",
    "sensor_id": "test-sensor",
    "requester_id": "test-requester",
    "source": "aidr",
    "detection_category": "Test Detection",
    "attributable_event_id": "test-attr",
    "concluding_event_id": "test-concl",
    "conviction_timestamp": datetime.datetime.utcnow().isoformat() + "Z",
    "mitre": json.dumps({"Tactic": {"UID": "AML.TA0001", "Name": "Test"}}),
    "severity": "high",
    "engine_name": "test_engine"
}]

print(f"Testing Azure Sentinel integration...")
print(f"Stream: {stream_name}")

# Authenticate
credential = ClientSecretCredential(tenant_id, client_id, client_secret)
client = LogsIngestionClient(endpoint=dce_endpoint, credential=credential, logging_enable=True)

try:
    response = client.upload(
        rule_id=dcr_immutable_id,
        stream_name=stream_name,
        logs=sample_log
    )
    print("✅ Test data sent successfully!")
    print(f"\nWait 5-10 minutes, then run this query in Log Analytics:")
    print(f"HiddenLayerAIDRStage_CL | where conviction_id == '{sample_log[0]['conviction_id']}'")
except Exception as e:
    print(f"❌ Error: {e}")

Install dependencies:

pip install azure-monitor-ingestion azure-identity

Run test:

python3 test_integration.py

Troubleshooting

Issue 1: Stream Not Configured Error

Error in DCRLogErrors:

The stream HiddenLayerAIDRStage_CL was not configured in the data collection rule

Cause: Incorrect table name in HiddenLayer Console Solution: ✅ Correct configuration:

Azure Portal table name: HiddenLayerAIDRStage
HiddenLayer Console table name: Custom-HiddenLayerAIDRStage_CL

❌ Incorrect configurations:

HiddenLayerAIDRStage_CL (missing “Custom-” prefix)
HiddenLayerAIDRStage (missing both prefix and suffix)
HiddenLayerAIDRStage_CL_CL (double suffix - table created incorrectly)

Issue 2: Authentication Failures

Error: No data in table Possible causes:

Incorrect Client ID or Client Secret
Expired Client Secret
Missing role assignment

Solutions: Verify credentials:

az login --service-principal \
  -u <CLIENT_ID> \
  -p <CLIENT_SECRET> \
  --tenant <TENANT_ID>

Check role assignment:

Go to Data Collection Rule → Access Control (IAM)
Verify app has “Monitoring Metrics Publisher” role
If missing, add the role (see Step 6)

Regenerate secret:

Go to App Registration → Certificates & secrets
Create new client secret
Update HiddenLayer configuration immediately

Issue 3: No Data Appearing

Symptom: No errors in DCRLogErrors, but no data in table Possible causes:

No convictions being generated
Integration not properly activated
Timing delay

Solutions: Check for detections:

HiddenLayer Console → AI Runtime Security → Detections
Look for detections in the last 24 hours

Trigger test detection:

Follow the steps in Testing the Integration

Diagnostic Queries

Check for Integration Errors

DCRLogErrors
| where TimeGenerated > ago(1h)
| where _ResourceId contains "<dcr-name>"
| summarize Count = count() by Message
| order by Count desc

Check Data Ingestion Rate

HiddenLayerAIDRStage_CL
| where TimeGenerated > ago(24h)
| summarize ConvictionsPerHour = count() by bin(TimeGenerated, 1h)
| render timechart

Check Conviction Severity Distribution

HiddenLayerAIDRStage_CL
| where TimeGenerated > ago(24h)
| summarize count() by severity
| render piechart

Find Recent High Severity Convictions

HiddenLayerAIDRStage_CL
| where TimeGenerated > ago(24h)
| where severity == "high" or severity == "critical"
| project TimeGenerated, conviction_id, detection_category, sensor_id
| order by TimeGenerated desc

Additional Resources

Microsoft Documentation

Quick Reference Card

Use this for easy reference:

Item	Value	Location
Tenant ID	`<your-tenant-id>`	Entra ID → Overview
Client ID	`<your-client-id>`	App Registration → Overview
Client Secret	`<your-secret>`	App Registration → Certificates & secrets
DCE Endpoint	`https://<endpoint>.<region>.ingest.monitor.azure.com`	DCE → JSON View → logsIngestion
DCE Resource ID	`/subscriptions/.../dataCollectionEndpoints/<name>`	DCE → JSON View → id
DCR Rule ID	`dcr-<guid>`	DCR → JSON View → immutableId
Stream Name	`Custom-HiddenLayerAIDRStage_CL`	Use in HiddenLayer Console
Table Name	`HiddenLayerAIDRStage_CL`	Log Analytics table

​Overview

​Architecture

​Prerequisites

​Setup Steps

​Step 1: Register Application in Azure Entra ID

​Step 2: Create Client Secret

​Step 3: Create Data Collection Endpoint

​Step 4: Create Custom Log Analytics Table

​Step 5: Get Data Collection Rule Details

​Step 6: Grant Permissions to App Registration

​Step 7: Configure HiddenLayer Console

​Configuration Summary

​Key Configuration Mapping

​Critical Field Clarifications

​Verification (Wait 5-10 minutes after configuration)

​Step 1: Create an AI Runtime Security detection

​Step 2: Check Azure for Errors

​Step 3: Verify Data Ingestion

​Testing the Integration

​Test Script

​Troubleshooting

​Issue 1: Stream Not Configured Error

​Issue 2: Authentication Failures

​Issue 3: No Data Appearing

​Diagnostic Queries

​Check for Integration Errors

​Check Data Ingestion Rate

​Check Conviction Severity Distribution

​Find Recent High Severity Convictions

​Additional Resources

​Microsoft Documentation

​Quick Reference Card

Overview

Architecture

Prerequisites

Setup Steps

Step 1: Register Application in Azure Entra ID

Step 2: Create Client Secret

Step 3: Create Data Collection Endpoint

Step 4: Create Custom Log Analytics Table

Step 5: Get Data Collection Rule Details

Step 6: Grant Permissions to App Registration

Step 7: Configure HiddenLayer Console

Configuration Summary

Key Configuration Mapping

Critical Field Clarifications

Verification (Wait 5-10 minutes after configuration)

Step 1: Create an AI Runtime Security detection

Step 2: Check Azure for Errors

Step 3: Verify Data Ingestion

Testing the Integration

Test Script

Troubleshooting

Issue 1: Stream Not Configured Error

Issue 2: Authentication Failures

Issue 3: No Data Appearing

Diagnostic Queries

Check for Integration Errors

Check Data Ingestion Rate

Check Conviction Severity Distribution

Find Recent High Severity Convictions

Additional Resources

Microsoft Documentation

Quick Reference Card