Overview

About Advantech Container Catalog (ACC)

Advantech Container Catalog is a comprehensive collection of ready-to-use, containerized software packages designed to accelerate the development and deployment of Edge AI applications. By offering pre-integrated solutions optimized for embedded hardware, ACC simplifies the challenge of software-hardware compatibility, especially in GPU/NPU-accelerated environments.

Feature / Benefit	Description
Accelerated Edge AI Development	Ready-to-use containerized solutions for faster prototyping and deployment
Hardware Compatible	Eliminates embedded hardware and software package incompatibility
GPU/NPU Access Ready	Supports passthrough for efficient hardware acceleration
Model Conversion & Optimization	Built-in AI model quantization and format conversion support
Optimized for CV & LLM Applications	Pre-optimized containers for computer vision and large language models

Container Overview

This container, GPU Passthrough on NVIDIA Jetson™, provides a ready-to-use environment with optimized AI frameworks, GPU passthrough, and industrial-grade reliability on NVIDIA Jetson™ platforms. It enables users to focus on developing AI applications on Advantech Edge AI systems accelerated by NVIDIA chipsets—eliminating the complexity of hardware setup and AI framework compatibility.

Key Features

Full Hardware Acceleration: Optimized access to GPU, NVENC/NVDEC, and DLA
Complete AI Framework Stack: PyTorch, TensorFlow, ONNX Runtime, and TensorRT™
Industrial Vision Support: Accelerated OpenCV and GStreamer pipelines
Edge AI Capabilities: Support for computer vision, LLMs, and time-series analysis
Performance Optimized: Tuned specifically for Advantech EPC-R7300 and more devices

Host Device Prerequisites

Item	Specification
Compatible Hardware	Advantech devices accelerated by NVIDIA Jetson™ - refer to Compatible hardware
NVIDIA Jetson™ Version	5. x
Host OS	Ubuntu 20.04
Required Software packages	*refer to below
Software Installation	NVIDIA Jetson™ Software Package Installation

Required Software Packages on Host Device

These packages are bound with NVIDIA Jetson™ version of the device. This container supports NVIDIA Jetson™ 5.x.

Component	Version	Description
CUDA® Toolkit	11.4.315	GPU computing platform
cuDNN	8.6.0.166	Deep Neural Network library
TensorRT™	8.5.2.2	Inference optimizer and runtime
VPI	2.2.7 or above
Vulkan	1.3.204 or above
OpenCV	4.5.4 with CUDA®: NO

Container Environment Overview

Software Components on Container Image

Component	Version	Description
CUDA®	11.4.315	GPU computing platform
cuDNN	8.6.0	Deep Neural Network library
TensorRT™	8.5.2.2	Inference optimizer and runtime
PyTorch	2.0.0+nv23.02	Deep learning framework
TensorFlow	2.12.0+nv23.05	Machine learning framework
ONNX Runtime	1.16.3	Cross-platform inference engine
OpenCV	4.5.0	Computer vision library with CUDA®
GStreamer	1.16.2	Multimedia framework

Container Quick Start Guide

For container quick start, including docker-compose file, and more, please refer to Advantech Container Repository

Supported AI Capabilities

Vision Models

Model Family	Versions	Performance (FPS)	Quantization Support
YOLO	v3/v4/v5 (up to v5.6.0), v6 (up to v6.2), v7 (up to v7.0), v8 (up to v8.0)	YOLOv5s: 45-60 @ 640x640, YOLOv8n: 40-55 @ 640x640, YOLOv8s: 30-40 @ 640x640	INT8, FP16, FP32
SSD	MobileNetV1/V2 SSD, EfficientDet-D0/D1	MobileNetV2 SSD: 50-65 @ 300x300, EfficientDet-D0: 25-35 @ 512x512	INT8, FP16, FP32
Faster R-CNN	ResNet50/ResNet101 backbones	ResNet50: 3-5 @ 1024x1024	FP16, FP32
Segmentation	DeepLabV3+, UNet	DeepLabV3+ (MobileNetV2): 12-20 @ 512x512	INT8, FP16, FP32
Classification	ResNet (18/50), MobileNet (V1/V2/V3), EfficientNet (B0-B2)	ResNet18: 120-150 @ 224x224, MobileNetV2: 180-210 @ 224x224	INT8, FP16, FP32
Pose Estimation	PoseNet, HRNet (up to W18)	PoseNet: 15-25 @ 256x256	FP16, FP32

Language Models Recommendation

Model Family	Versions	Memory Requirements	Performance Notes
DeepSeek Coder	Mini (1.3B), Light (1.5B)	2-3 GB	10-15 tokens/sec in FP16
TinyLlama	1.1B	2 GB	8-12 tokens/sec in FP16
Phi	Phi-1.5 (1.3B), Phi-2 (2.7B)	1.5-3 GB	Phi-1.5: 8-12 tokens/sec in FP16, Phi-2: 4-8 tokens/sec in FP16
Llama 2	7B (Quantized to 4-bit)	3-4 GB	1-2 tokens/sec in INT4/INT8
Mistral	7B (Quantized to 4-bit)	3-4 GB	1-2 tokens/sec in INT4/INT8

DeepSeek R1 1.5B Optimizations Recommendations:

Supports INT4-8 quantization for inference
Best performance with TensorRT™ engine conversion
Typical throughput: 8-12 tokens/sec in FP16, 12-18 tokens/sec in INT8
Recommended batch size: 1-2 for real-time applications

Supported AI Model Formats

Format	Support Level	Compatible Versions	Notes
ONNX	Full	1.10.0 - 1.16.3	Recommended for cross-framework compatibility
TensorRT™	Full	7.x - 8.5.x	Best for performance-critical applications
PyTorch (JIT)	Full	1.8.0 - 2.0.0	Native support via TorchScript
TensorFlow SavedModel	Full	2.8.0 - 2.12.0	Recommended TF deployment format
TFLite	Partial	Up to 2.12.0	May have limited hardware acceleration

Hardware Acceleration Support

Accelerator	Support Level	Compatible Libraries	Notes
CUDA®	Full	PyTorch, TensorFlow, OpenCV, ONNX Runtime	Primary acceleration method
TensorRT™	Full	ONNX, TensorFlow, PyTorch (via export)	Recommended for inference optimization
cuDNN	Full	PyTorch, TensorFlow	Accelerates deep learning primitives
NVDEC	Full	GStreamer, FFmpeg	Hardware video decoding
NVENC	Full	GStreamer, FFmpeg	Hardware video encoding
DLA	Partial	TensorRT™	Requires specific model optimization