Hi, my name is Harpreet.
Watch agent - my bot to analyze process failure and suggest solutions and then wait for the human to give a go ahead to implement the solution
I built an AI agent to monitor and assist with the operational layer of my quantitative research platform. My platform runs multiple Apache Airflow pipelines that ingest and transform financial data from various vendors into self-hosted time-series databases. As the number of workflows grew, diagnosing failures became time-consuming and cognitively expensive.
Beyond infrastructure, the research layer of the platform also uses agentic workflows. These agents assist with parsing financial statements, extracting risk factors from SEC filings, performing sentiment analysis on disclosures, and supporting backtesting workflows. The system is designed so that AI augments both operational reliability and research productivity.
What the human can now do that they couldn’t before?
On the infrastructure side, when a pipeline fails, instead of manually combing through logs and tracing dependencies, the human now receives a structured diagnostic summary: root cause, suggested remediation, confidence level, and risk considerations. The human shifts from log investigator to decision-maker.
On the research side, the human can analyze far more information than before. Instead of manually reading lengthy filings or restructuring financial statements for modeling, the AI pre-processes and structures this data. It highlights risk factors, summarizes management commentary, identifies sentiment shifts, and prepares datasets for backtesting. The human moves from information gathering to higher-level interpretation and strategy design.
What AI is responsible for?
Operationally, AI monitors DAG executions, retrieves logs, interprets failure traces, detects historical patterns, and proposes corrective actions. It classifies issues such as schema mismatches, missing files, API changes, or transformation errors.
On the research side, AI parses structured and unstructured financial data, extracts relevant features, summarizes long documents, flags risk language, assists in generating backtest-ready datasets, and supports hypothesis exploration.
In both cases, AI is responsible for analysis, pattern recognition, summarization, and structured proposal generation.