The following licenses and keys are required for deploying Model Scanner CLI in Hybrid mode. If your organization doesn't have a license, contact HiddenLayer for more information.
Model Scanner License Key: HiddenLayer Support will provide you with a license key. This key is required to start the LLM proxy container, and it will not run without a valid key. The license can be set as an environment variable and the installer will not run without the license being set as a value.
Credentials to download Model Scanner container: Credentials for the HiddenLayer container repository are required to download the appropriate images. These can also be obtained from HiddenLayer Support or from your HiddenLayer technical contact.
API Client ID and Client Secret: HiddenLayer API Client ID and Client Secret to interact with the AISec Platform Console. Get these from the Console or your Console Admin.
- Link to the US Console
- Link to the EU Console
The following licenses and keys are required for deploying Model Scanner CLI in Disconnected mode. If your organization doesn't have a license, contact HiddenLayer for more information.
- Model Scanner License Key: HiddenLayer Support will provide you with a license key. This key is required to start the LLM proxy container, and it will not run without a valid key. The license can be set as an environment variable and the installer will not run without the license being set as a value.
- Credentials to download Model Scanner container: Credentials for the HiddenLayer container repository are required to download the appropriate images. These can also be obtained from HiddenLayer Support or from your HiddenLayer technical contact.
Disconnected mode does not require an API Client ID and Secret.
The following tools are required for deploying the Model Scanner CLI locally on your system.
Computer system: With a minimum of 8 CPU cores and 16GB memory (most modern laptops).
- Performance will vary based on resources. Typically, the more resources the better the performance.
Docker Desktop: Docker Desktop is used to deploy the container.
kubectl: The official Kubernetes CLI tool, used to issue commands to your Kubernetes cluster.
The tool requirements apply to both Hybrid and Disconnected modes.
The following are resource management examples for the Model Scanner k8s Docker job, based on the model type. The below stats are with default behavior.
- The examples below do not include the time it takes to download the model file because download speeds can vary.
- A compute system with 12 CPU threads, 16GB RAM, and 100GB disk space was used to obtain these stats. These are not system requirements but are provided as a reference.
| Model Type | Max RSS RAM | Largest File Tested | Duration |
|---|---|---|---|
| GGUF | 40 MB | 4 GB | 16.6 sec |
| Keras | 375 MB | 3.2 GB | 49.7 sec |
| Nemo | 130 MB | 2.31 GB | 24.8 sec |
| Numpy | 92 MB | 11.73 GB | 3 min 11 sec |
| Onnx | 2.3 GB | 1.52 GB | 10.5 sec |
| Pickle ✳ | 150 MB | 10 GB | 1 min 35 sec |
| PyTorch | 65 MB | 9.26 GB | 1 min 33 sec |
| RDS | 52 MB | 3.94 GB | 38 sec |
| Safetensor | 260 MB | 9.26 GB | 1 min 28 sec |
| skops | 41 MB | 1.21 GB | 41 sec |
✳ Memory usage depends on the Pickle file encoding. A minimum of 16 GB RAM is recommended.