PIG-MENT

Computer Vision applied to biomedicine

Hours of video analyzed in minutes: the animal's behavior measured and compared throughout its recovery, without anyone having to see it.

Pigment animal estadisticas
Pigment microscopio

The Germans Trias i Pujol Research Institute (IGTP) is evaluating the recovery of study animals using video footage. Until now, each hour of recording required an hour of a person's attention, and the result depended on who was watching. With PIG-MENT, the platform developed by Mecexis, that hour of video is transformed into two minutes of calculation and a set of identical behavioral metrics for each run, organized by animal, by time point in the study, and by treatment group, ready for statistical analysis. The research team stops watching videos and starts analyzing data.

Biomedical research that needed to measure, not describe

A leading institute and a question without a numerical answer

The IGTP is a public research center located in Badalona, ​​associated with the Germans Trias i Pujol Hospital and part of the Can Ruti biomedical campus. It is a CERCA center, accredited as a center of excellence by the Carlos III Health Institute. Its team works with a porcine animal model to study recovery after surgery: how each animal progresses in the following days and whether a treatment modifies that progress compared to the control group.

The data already existed. Fixed cameras above each pen record morning, afternoon, and evening sessions, before and after the intervention. What was missing was a way to extract objective values ​​from those hours of video footage to create statistics, without anyone having to sit and watch them.

Pigment comparison charts

Hours of human dedication for every hour of recording

A bottleneck that grows with each day of study

Post-surgical assessment was done by observation: one person watched the recordings and scored the animals' behavior. The cost is linear and doesn't decrease: each hour of video costs one hour of attention, and a studio records each animal several times a day for days, before and after the procedure. The volume of video grows faster than any team can keep up, and what isn't seen can't be measured.

Added to this cost is a quality problem. Two observers don't obtain the same result, and what emerges from the observation are descriptions and scales, not measurements: nothing with which to detect a subtle trend or rigorously compare two groups. The question that the IGTP brought to Mecexis was twofold: can this task be automated, and can it be done in a way that produces a better result than the manual method, not just one that is cheaper?

The answer wasn't obvious. Wide-angle cameras distorting the image, partially visible corrals, ropes and other obstacles crossing in front of the animal, and no date recorded in the video file. And one specific problem ruled out the usual approach: methods that detect the animal through movement by differentiating it from the background "absorb" it when it remains still, precisely at the moment when the resting time needs to be measured.

Pigment results

Computer Vision to automate the processing of recordings, without losing the team's judgment

PIG-MENT consists of two parts that work together: a chain of computer vision algorithms that extracts metrics from video without human intervention, and a web application where the research team manages, reviews, and compares the entire process. The machine does the repetitive work that previously took hours; people retain the decisions that require judgment, such as validating the geometry of each camera or interpreting the results.

  • Three metrics from a single video

    Given a video file as the sole input, the system returns three measurements without human intervention. Circling counts the animal's complete turns in each direction and detects whether it consistently favors one side—the most direct neurological indicator following intervention. The occupancy map shows where the animal spends its time within the pen, in actual centimeters, and how much time in each area of ​​interest. Rest time measures what fraction of time the animal remains motionless and how that time is distributed. Each video also includes its own quality indicators and control figures, ensuring that no single metric is ever presented in isolation.

  • Optimized for conventional CPUs, without GPU

    The animal is located by its warm color, using segmentation, contours, and image timing. This decision has very positive practical consequences: not a single video needed to be tagged, no GPU is required, the infrastructure cost is low and predictable, and each result can be explained by the algorithm. Furthermore, it resolves the rest-by-construction problem: the animal is pink whether it is still or moving.

  • Actual centimeters, comparable between corrals and chambers

    Each camera is calibrated once using a calibration pattern to correct lens distortion, and each recording automatically detects the four corners of the corral floor. With the actual corral dimensions, every measurement is expressed on the same plane in centimeters. Two different corrals or two camera positions produce comparable numbers, and the camera height isn't even configured: it's automatically determined from the validated geometry.

  • The video is processed once; everything else takes seconds.

    The video analysis, which is the most resource-intensive part, is performed only once and records the animal's position, area, and orientation in each frame. Recalculating a metric, correcting the scale, or recomputing the zones works on this data and takes only seconds. When the disk is full, the oldest footage can be deleted without loss: the results remain reproducible.

  • A platform that knows the studio

    The web application guides the entire workflow: uploading the clip (directly to storage, bypassing the server, for files of several gigabytes), describing it, validating the camera's position against a reference image, processing it in the background, and reading the results. The recording date is read from the clock printed on the image itself, as the camera does not write it to the file. Phase, days since the intervention, and time slot are always derived from two dates, so correcting one corrects all pages.

  • The layer of interpretation: from isolated video to comparison

    The scientific value lies not in the video itself, but in the comparison. The animal's evolution is analyzed by combining recordings from sessions on a real-time timeline, starting from the intervention, and showing the change in each metric relative to its pre-operative state. The comparison tool compares groups of animals with a reference group and calculates the difference of differences, answering the study's question: Did the treatment alter the animal's evolution? Evidence filters remove excessively short clips from the median and alert you on screen. The tool presents the data; the researcher draws the conclusions.

  • Data ready for statistics, with its codebook

    Each export is a single file containing fourteen tables linked by identifiers, in Excel, CSV, and Parquet formats, plus the figures, trajectories, and a README file generated from the code itself, defining each column, each metric, and the rules used for calculation. It is possible to hide the treatment group from the results so researchers can work without being influenced by them.

  • Production-ready architecture

    Django and PostgreSQL, background processes running on two queues so interaction never waits behind a batch of videos, S3-compatible object storage, Dockerized infrastructure, automated deployment, error tracking, and a suite of synthetic trajectory tests that define the algorithm's behavior. The same discipline that Mecexis applies to every project, applied to a scientific measuring instrument.

First prove it was possible, then build the product

A prototype for deciding, a platform for working

The project began with a feasibility question, not a platform. The first phase was designed as a prototyping process: analyzing the problem, reviewing the state of the art and existing tools, and building the computer vision chain using real IGTP recordings. The goal was to determine, based on data, whether the project was viable. The results were compared with what a person sees on screen, such as counting the number of turns an animal makes or the difference between a calm and an active one, and with the speed measured on a conventional CPU. Only after verifying that the foundations on which the project was built were solid did they move forward.

The second phase transformed the prototype into a product: the web platform with the study's data model, background processing, and export capabilities, delivered to the research team as a beta version for real-world use. The third phase built upon the requirements of a month's use, added the comparison layer, and deployed the system to production. Each phase had a defined scope and was delivered and validated before the next phase began, ensuring that the client could make informed decisions at each stage based on the previous results.

Working with real footage from day one set the course. Ropes cutting across the animal's silhouette, individuals paler than others, reflections, neighboring corrals peeking into the frame, videos without recorded dates: every obstacle appeared in the studio's recordings during prototyping, when correcting it was inexpensive, and it was resolved in the relevant part of the process without affecting the others. And where judgment is needed, the team provides it: the geometry of each camera is validated by a person on the image, using the system's automatic suggestion as a starting point. The tool proposes, and the person confirms.

Pigment areas recording

Drastic reduction in manual observation time

People's time, dedicated to what only people can do.

  • 1h → 2min

    Video review time

  • 0h

    Viewing time to obtain metrics

  • 100%

    Frames with the animal located

A database designed to extract more metrics

New metrics without rewriting existing ones

The processing chain was designed to grow: each metric is an independent algorithm on a common base of signals, so adding a new one is like adding a piece to the end without checking the previous ones.

On the horizon are the automation of capture, with the cameras delivering the clips directly and the platform cutting them by session, and the extension to other facilities: lens calibrations are uploaded by name, so that a second center with its own cameras works without forking the code.

Pigment comparison