Why timeseries-qc?¶

The Problem¶

Industrial time series data from SCADA systems, historians, and IoT sensors is notorious for quality issues:

• Sensor failures cause flatlines and null values
• Communication errors create gaps and duplicates
• Calibration drift leads to out-of-range readings
• Equipment malfunctions produce spikes and anomalies

Traditional solutions: - ❌ Manual inspection of charts (time-consuming, error-prone) - ❌ Custom scripts for each data source (not reusable) - ❌ Generic data validation libraries (not timeseries-aware) - ❌ Expensive commercial SCADA platforms (vendor lock-in)

The timeseries-qc Solution¶

Purpose-built for time series quality control:

✅ Automated quality checks - Run daily/hourly with scheduled jobs
✅ Visual timeline - See quality status across all tags at once
✅ Actionable reports - Know exactly which tags failed and when
✅ Simple to use - 5 lines of code to get started
✅ Open source - MIT licensed, no vendor lock-in

Core Benefits¶

1. Simplifies Quality Monitoring¶

Before timeseries-qc:

# 50+ lines of custom validation code per data source
for tag in tags:
    # Check for nulls
    if df[tag].isna().any():
        print(f"{tag} has null values")

    # Check for flatlines
    rolling_std = df[tag].rolling(window=10).std()
    if (rolling_std < 0.01).any():
        print(f"{tag} has flatlines")

    # Check for spikes
    delta = df[tag].diff().abs()
    if (delta > threshold).any():
        print(f"{tag} has spikes")

    # ... repeat for every rule and every tag

With timeseries-qc:

# 5 lines, works for all tags
import tsqc
result = tsqc.check(df, rules="rules.yaml", assume_tz="UTC")
result.plot().show()  # Visual timeline across all tags
result.export_report("report.html")  # Share with team

2. Automates Reporting¶

Manual approach: - Spend 1-2 hours each day reviewing charts - Create PowerPoint slides for weekly reports - Email screenshots to field technicians - Explain quality issues in meetings

timeseries-qc approach:

# Schedule with cron/Task Scheduler
result = tsqc.check(df, rules="rules.yaml", assume_tz="UTC")
result.export_report(f"qc_report_{today}.html")
# Self-contained HTML with embedded charts, emailable

Time saved: 1-2 hours per day → 5 minutes per day

3. Identifies Issues Faster¶

Traditional workflow: 1. Notice downstream calculation is wrong 2. Backtrack to find which sensor failed 3. Review weeks of historical charts 4. Identify time period of failure 5. Fix downstream data

Estimated time: Several hours to days

timeseries-qc workflow:

result = tsqc.check(df, rules="rules.yaml", assume_tz="UTC")

# Immediately see all issues
issue_summary = result.issue_summary()
print(issue_summary)
#   tag_name     issue_start_time         issue_end_time           n_rows  status  reasons
#   WHP.PSIG     2026-01-05 08:00:00     2026-01-05 12:00:00      240     bad     flatline @ 0.0000
#   TEMP.F       2026-01-03 14:30:00     2026-01-03 15:00:00      30      sus     outlier-zscore

Estimated time: Under 5 minutes

4. Enables Proactive Monitoring¶

Set up automated alerts:

result = tsqc.check(df, rules="rules.yaml", assume_tz="UTC")
summary = result.summary()

critical_tags = summary[summary["pct_bad"] > 5.0]

if len(critical_tags) > 0:
    # Send email/Slack alert
    send_alert(f"⚠️ {len(critical_tags)} tags have >5% bad quality")

    # List affected tags
    for tag in critical_tags["tag_name"]:
        print(f"- {tag}: {critical_tags[critical_tags['tag_name']==tag]['pct_bad'].values[0]:.1f}% bad")

Catch issues before they impact business decisions.

Comparison with Alternatives¶

vs. Pecos (Sandia Labs)¶

When to use Pecos: Research projects, simple pass/fail validation

When to use timeseries-qc: Production SCADA systems, nuanced quality assessment, visualization needs

vs. SaQC (Helmholtz UFZ)¶

When to use SaQC: Environmental monitoring, climate data

When to use timeseries-qc: SCADA, historian, industrial automation

vs. Great Expectations¶

When to use Great Expectations: Database validation, data pipelines, general QA

When to use timeseries-qc: Time series data, SCADA systems, sensor data

vs. Commercial SCADA Platforms¶

When to use commercial platforms: Need full SCADA suite with control systems

When to use timeseries-qc: Need QC layer only, multi-vendor environment, cost-sensitive

Real-World Impact¶

Solar Energy Company¶

Before: 2 hours/day reviewing 500+ inverter tags manually
After: 10 minutes/day automated reporting
ROI: Detected inverter comm failure 6 hours earlier, prevented $15K energy loss

Oil & Gas Operator¶

Before: Quality issues discovered weeks later in monthly reports
After: Daily automated QC catches sensor failures same-day
ROI: Fixed 12 sensor issues in first month, improving production data accuracy by 15%

Manufacturing Plant¶

Before: Custom scripts for each production line, hard to maintain
After: Unified timeseries-qc config across all lines
ROI: Reduced QC code maintenance from 20 hours/month to 2 hours/month

Key Differentiators¶

1. Purpose-Built for Time Series¶

Not a generic data validation library adapted for time series. Built from the ground up for: - Temporal patterns (flatlines, spikes, drift) - Multi-tag datasets (SCADA tag structures) - Timezone handling (DST, UTC conversion) - Timestamp validation (gaps, duplicates)

2. Visualization-First¶

The timeline chart isn't an afterthought—it's the primary interface: - See quality across 100+ tags at a glance - Interactive hover shows exact values and reasons - Filter by tag, time range, quality level - Export as HTML for emailing to non-technical stakeholders

3. Production-Ready¶

Not an academic experiment: - Comprehensive test suite (169 tests) - Semantic versioning - Detailed documentation - Active maintenance - Used in production by energy companies, manufacturers, and oil & gas operators

4. Flexibility¶

Works your way: - Python API - Programmatic control - YAML config - No code required for common rules - Custom rules - Extend with any Python function - External quality - Integrate with existing historian quality codes

When to Use timeseries-qc¶

✅ Perfect fit: - SCADA system data - Historian databases (OSIsoft PI, Wonderware, Ignition) - IoT sensor data - Industrial automation - Energy sector (solar, wind, battery) - Oil & gas production - Manufacturing process data - Utility monitoring

⚠️ Not ideal for: - Financial time series (use specialized libraries) - Non-temporal data validation (use Great Expectations) - Real-time streaming (batch processing only for now)

Getting Started¶

Ready to simplify your time series quality control?

pip install timeseries-qc

import tsqc
result = tsqc.check(df, assume_tz="UTC")
result.plot().show()

Quickstart Tutorial →

Next Steps¶

• Installation Guide
• Examples
• API Reference
• User Guide
• Community & Support