Report Generator

Skygauge Robotics - Fall 2022

What was this project?

This is a project to create scripts that automate the process of converting logs into more readable formats for Skygauge. The report would consist of summary pages that summarized the contents of the report as well as a detailed tables and images for each portion of the inspection. These reports varied in length based on the length of the inspection, but would usually range from 20 to 70 pages per report. Previously, these reports were done manually and took multiple hours. Each detailed page would require manually scrolling through tables of logs hundreds of thousands of entries (rows) long with hundreds of attributes (columns). The data would then need to be manually filled into template documents and aggregated into a final document. Overall the task was vary daunting to do manually and so the project was started to automate this process. A previous intern started the project, reaching a point where the script could read a manually created CSV file summarizing the logs to fill in tables from a template document. It could then combine the individual pages into a list of tables on one document.

What were the goals of this project?

The goal of this project was to speed up this process using automation. Doing each report manually would take multiple hours of labour to create, but the task was mostly repetitive. Using a script to automate the report creation process and only requiring human interaction for the more subjective parts of data selection would significantly speed up report generation. In addition, having a script automatically note abnormalities in the data could speed up the manual review process that each report undergoes by highlighting areas where data was not as certain and relatively more likely to contain incorrect information.

Who was involved?

Picking up from the previous intern's work, I had to automate the creation of the report summary CSV that was previously done manually. I was also responsible for additions to the script to automate the generation of cover pages, extract as many valid logs from occasionally corrupted data, highlight dubious data likely to be incorrect and generate other metrics deducible from the data for easier analysis from the report. Analysis of the raw data itself and determining best data was done by another engineer, where then that data was passed to my script to be turned into reports.

How did it turn out?

The end result was a python script that could process a flight sets logs by dragging and dropping the folder of logs into the scripts terminal and following some instructions on screen. It would shorten the time taken for creating a report from hours to about 5-10 minutes depending on the length of the flight set. Some extra quality of life features were also added such as an error highlighter that would highlight areas of concern in the report when numbers would indicate that a particular contact may have incorrect readings. When reading through a report for errors, the script could also calculate some other statistics to highlight any outliers in the data.