Transaction Processing Performance Council Introduces AI Benchmark (TPCx-AI)
TPCx-AI makes use of a diversified dataset and was created with the goal of being flexible to a wide array of scale factors.
FREMONT, CA: Industry users are often required to be nourished with relevant, objective performance data. To achieve this feat, Transaction Processing Performance Council (TPC), a non-profit corporation, recently launched its artificial intelligence benchmark (TPCx-AI). TPCx-AI is the first of its kind industry-standard, vendor-neutral benchmark for assessing real-world, end-to-end AI and ML scenarios and data science use cases. TPCx-AI makes use of a diversified dataset and was created with the goal of being flexible to a wide array of scale factors.
“The TPCx-AI benchmark is the result of collaboration between talented engineers and researchers at some of today’s leading AI organizations,” said Hamesh Patel, Chair of the TPCx-AI committee and principal engineer at Intel Corporation. “It is designed to emulate real-world examples of organizations that use a variety of production ready data science pipelines–including both AI and ML approaches–and is now widely available to anyone who would like to download and run it. We look forward to feedback as industry experts, academics and others interested in benchmarking system performance begin to use it.”
TPCx-AI is a general-purpose data science system that:
• Generates and processes huge volumes of data.
• Trains preprocessed data to generate realistic machine learning models.
• Using the produced models, conducts accurate insights for real-world client scenarios.
• Capable of scalability in large-scale distributed configurations.
• Allows configuration modifications with greater flexibility to suit the demands of the ever-changing AI landscape.
The benchmark also measures end-to-end time to give insights for specific use cases and throughput metrics to simulate multi-user environments for specific hardware, operating system, and data processing system configuration under a controlled, complex, multi-user AI or machine learning data science workload.