“Set it and forget it” is a popular saying on many late night infomercials. Take some new cooker, throw your food into it, push a few buttons and then a few hours later you have amazing gourmet meals with no effort. At least that’s the theory.
In the business world, many people have begun treating their algorithms the same way. They create these elaborate rules for metrics, benchmarking and scoring that will assess a thousand variables to come up with the perfect rank. The best will even apply the probability curves around the score that is generated. Today they may even give results that make sense.
Time is fickle. As time passes, conditions change. The rules that governed a process no longer apply because people begin moving back to cities or technologies change the way that work is done or home officing continues to pick up or local policies change the way that financials are calculated. Something always changes.
But this change is often not handled well in algorithms. Often, the team that builds them puts a pin in them and then moves on to the new shiny toy letting the old one run with no supervision. What this really means is that there is no one around to catch it when it stops returning valid answers. To a layperson it may seem like good numbers – everything worked, the data is all there, the results are consistent with what was previously calculated – yet now the answers are no longer statistically valid for some reason.
Shelf-life is a mandatory concept within the perishable food space. It should also be a concept within the data science space. Data can go stale over time much like algorithms can no longer be applicable.