![]() CPU bottlenecks addressed by GPU accelerationĬV-CUDA provides highly optimized, GPU-accelerated kernels as standalone operators for computer vision processing. This can result in significant annual cloud cost savings on the order of hundreds of millions of USD, and hundreds of GWh of annual savings in energy consumption in data centers. In this post, we demonstrate the benefit of using CV-CUDA to enable end-to-end GPU acceleration for a typical AI computer vision workload achieving ~5x to up to 50x speedup in overall throughput. For more information, see the keynote announcement at NVIDIA GTC Fall 2022. The library provides a specialized set of GPU-accelerated computer vision and image-processing kernels as standalone operators to easily implement highly efficient pre- and post-processing steps of the AI pipeline.ĬV-CUDA can be used in a variety of common computer vision pipelines, such as image classification, object detection, segmentation, and image generation. Conventional workflow for AI computer vision pipeline CV-CUDA optimizationĬV-CUDA is an open-source library that enables you to build efficient cloud-scale AI computer vision pipelines. The decoding and encoding processes that are typically part of an AI image or video processing pipeline can also be bottlenecked on the CPU, affecting the overall performance.įigure 1. This leads to bottlenecks in the performance of the entire AI pipeline. While developers may use NVIDIA GPUs to significantly accelerate the AI model inference in their pipelines, pre– and post-processing are still commonly implemented with CPU-based libraries. Common operators in these steps include the following: Given these trends, there is a critical need for building high-performance yet cost-effective computer vision workloads.ĪI-based computer vision pipelines typically involve data pre- and post-processing steps around the AI inference model, which can be 50–80% of the entire workload. The shift from still images to video is also now becoming the primary component of consumer Internet traffic. ![]() However, the compute cost of these workloads is growing too, driven by demand for increased sophistication in the processing. The use cases include image understanding, content creation, content moderation, mapping, recommender systems, and video conferencing. Real-time cloud-scale applications that involve AI-based computer vision are growing rapidly.
0 Comments
Leave a Reply. |