Lightup Data
Proactively Detect Bad Data with Faster Data Quality Checks

Around three and a half years ago, McDonald’s set out to build its first loyalty program. This program aimed to enhance customer engagement by personalizing experiences and rewards based on purchasing behavior. Given their large business scale—over 14,000 restaurants in the U.S. and more than 40,000 globally—and the massive amount of data they handle, this initiative was far from straightforward. Every transaction, from buying a burger to placing an order through various channels, had to be recorded accurately and promptly across their entire data infrastructure. To achieve this, they required developer support and data domain expertise since developing manual data quality checks with legacy tools was time-consuming and resource-intensive.

As a leading developer of modern no-code deep data quality platforms, Lightup Data offered the very solution to the challenge at hand. The company provided a purpose-built platform that helped McDonald’s wind down and retire its legacy data quality tools. The goal was clear—to ensure that every single endpoint, every point-of-sale system, every kiosk, and all the various ordering methods—five different ways in total—were all feeding back transactions without failures. The platform helped McDonald’s dramatically reduce the time required to set up data quality checks to minutes, which once took months. This efficiency was crucial to the success of the loyalty program, enabling McDonald's to maintain high data integrity and enhance overall operations. Many business owners have since witnessed the same transformation as McDonald’s, gaining the ability to manage data quality checks more quickly and easily on their own, thanks to Lightup.

What Makes Lightup Achieve This?

“Lightup’s no-code, automated anomaly detection tool makes monitoring data quality and availability checks 10 times easier,” says Manu Bansal, CEO and co-founder of Lightup.

The solution is universal. It can be deployed by any large enterprise that handles fast-moving, high-volume data for their business operations. For example, handling consumer transactions, processing healthcare claims, managing large fleets of equipment with continuous IoT data, or maintaining complex manufacturing supply chains. Lightup provides deep checks for any kind of scenario before bad data creates any business loss. This platform’s ability is driven by a key innovation—a pushdown architecture. The platform performs data quality checks directly where the data already resides, like in a database or data stack. This allows users to specify checks as if they were for small datasets. Later, the system converts these checks into implementations that work at scale. Once the scale issue is fixed, anomaly detection becomes more straightforward with the platform’s automated anomaly detection capabilities.

“Our mission is to make data quality checks accessible and straightforward, allowing anyone who works with data to perform these tasks. This model has proven successful, with hundreds of users across organizations authoring checks on their own,” says Bansal.

The goal is to ensure that data quality checks are seen as an everyday operation without the need for an expert. If one can work with data, they must be able to ensure its quality.

That is why the company’s approach to data quality embraces democratizing data operations—empowering various stakeholders across the organization to engage with and use data effectively. The platform allows data analysts, analytics engineers, line-of-business users, and data scientists to create and deploy their data quality checks at scale without requiring specialized skills. It is also designed so that even non-technical business users can define, manage, and scale data quality checks easily. They can understand what good data looks like for their business without needing to write complex code or grappling with machine learning technicalities. All that is taken care of by Lightup automatically.

Enterprise Data Security and Safety

Large enterprises cannot afford data leakage and insufficient controls over who can access their data, even when performing essential quality checks. Compliance with regulations like the General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA) is non-negotiable too. That being the case, security and safety assurance for enterprise data is a vital concern for Lightup. This is achieved through dual-plane architecture. The data plane, where all data processing occurs, is deployed in the client’s virtual private cloud (VPC). Meanwhile, a control plane resides in Lightup’s cloud environment to manage the data plane instance. This setup combines the best of both worlds. The client benefits from a managed environment akin to a SaaS service while data processing remains as secure as an on-premises deployment. Ultimately, this helps to maintain data jurisdiction, keeping all data processing within the client’s environment.

Lightup’s no-code, automated anomaly detection tool makes monitoring data quality and availability checks 10 times easier.

The architecture has already proven to be powerful, particularly in regulated environments, including healthcare compliance and data quality assurance for financial services. The company has further invested in role-based access control (RBAC) to delineate roles, assign responsibilities, and control data visibility. For example, personally identifiable information (PII) or protected health information (PHI) fields can be restricted to individuals with the highest clearance levels. Lightup's platform ensures no PII is exposed in the results, even when running data quality checks on unmasked PII.

Lightup’s collaboration with clients doesn’t end with just offering the platform but extends to helping them establish best practices around using the tool. This entails assisting them in building processes with their people or organizing their teams to effectively operate the product.

What’s Next?

The company is inspired to expand its coverage for end-to-end data quality after witnessing the platform’s success and demand from clients. Now, the goal is to enable data quality checks at any stage in the pipeline, no matter what data stack they use—for data that's structured, unstructured, at rest, or in motion. Lightup is also focused on ease of use, continuously enhancing out-of-the-box data quality checks to make them simple enough to be deployed with just a few clicks. These checks can be tailored to specific industries or use cases. The company’s focus is further extended to investing in solutions for automatic remediation, which will be available within the next 6 to 9 months. Another emphasis is on developing data quality solutions tailored for AI and large language model (LLM) applications. Lightup aims to make it easier to write checks, possibly through natural language interfaces or by inferring what a check might look like and providing recommendations by leveraging AI capabilities.

This commitment to innovation and customer-centric solutions solidifies Lightup's position as a leader in modern data quality platforms, empowering organizations to maintain data integrity and drive operational excellence effortlessly.