How to efficiently reverse fill or bfill operations

So you’ve done it, you’ve got a nice time series with helpful features in a pandasDataFrame. Maybe you’ve used pd.ffill()or pd.bfill() to fill in empty time steps using the previous or next value and perform analysis or feature extraction on your full series.

Your
Your
Photo by JJ Ying on Unsplash

What do you do?

We faced this problem today at Mansa, where we were saving hundreds (if not thousands) of unnecessary rows after our pre-processing pipeline was completed.

Since we deal with financial data, we want to be able to tell the balance for an account at any point in time and to calculate statistics over number…

Gautier Dagan

MSc. in AI @ UVA | Previously, LD15 @ EF and 1st Employee + ML Engineer @ Mansa

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store