Datagaps: 'Validating' Data Quality and Accuracy

“Working at Oracle Corporation for over a decade on complex products and one of the largest Oracle Business Intelligence implementations was an amazing experience,” says Narendar Yalamanchilli, the Founder and CEO of datagaps—a Sterling, VA based company that specializes in providing testing automation solutions. Frustration due the lack of standard methodologies and tools to test reporting systems with huge volumes of data critical for businesses inspired him to come up with the idea of Datagaps. Two years after the company was founded, Adithya Buddhavarapu, an experienced Product Strategist and Oracle alumnus joined Naren as a partner to drive datagaps forward and realize the vision of solving the data testing problem. Currently, they have a fast growing customer base and are excited to be working with a number of the Fortune 500 companies.

As per Naren and Adithya, while most aspects of the design of data integration and business intelligence systems have received considerable attention, existing techniques do not provide for an automated way of testing data integration systems and this compelled him to establish Datagaps, which addresses this issue. ETL Validator and BI Validator—solutions that are designed to help customers automate the testing of Oracle Business Intelligence and other data warehousing projects—are the flagship products of the firm that emphasizes on building trust in enterprise data and reporting.

ETL Validator is a data testing tool that greatly simplifies testing of Data Integration projects and is built specifically for automating the testing of Data Warehouse projects. Running on the home grown patented ELV Architecture, this tool supports extraction of data from multiple sources such as relational databases, big data, XML, flat files and OLAP/BI. With ETL Validator, the clients can test for Data Validity by verifying that the data conforms to standards and range of values; ensure Data Accuracy by making sure that the data values are closer to real world values; and perform metadata testing by ensuring that the data structures conform to the data dictionary standards across the source and target systems.

The tool can help detect defects prior to production and can drastically reduce the costs associated with BI/ETL software projects. In addition, the automation capabilities of the tool can enable quality assurance teams to run in a lean manner with the right number of testers focusing on the right set of tasks. On the contrary, BI Validator is used by business analysts, testers and developers to automate the regression and performance testing of dashboards and analysis, thus avoiding manual eye balling of dashboard pages for regression testing.

Shedding light on the biggest success story of Datagaps, Naren gives an example of how his company helped a client who had difficulties in migrating about 1000 tables from Oracle database to Netezza. The client wanted a tool that not only compares millions of records in a reasonable amount of time, but also automatically creates test cases and then run them within the shortest possible time. And they opted for ETL Validator—Datagaps’s automation tool. Using ETL Validator, the clients were able to compare millions of records and with the tool’s built-in wizard they were able to create lot of test cases, in a very short period of time.

Moving forward Datagaps aims to dig deeper into the analytics (Big Data analytics) space and solve the various issues affecting the industry. Adi, who currently leads Marketing Strategy and Business Development, says that the company is also emphasizing on building better product designs and plans to continue to support new data sources and new BI platforms, and in addition provide greater customer experience that revolves around product and service.

We emphasize on building trust in enterprise data and reporting