How to Detect Outliers in a CSV Dataset

Two Common Detection Methods

A third, simpler option: fixed percentile cutoffs (e.g. flag anything below the 1st percentile or above the 99th). Crude, but useful when you just want to trim the extreme tails without a statistical model.

Choosing a Method Based on Your Data's Shape

Z-score works well when the data is roughly bell-shaped, income or transaction-amount data usually isn't (it's right-skewed, with a long tail of high values), which makes z-score prone to flagging real, valid high values as outliers. IQR handles skewed distributions more gracefully because it's based on ranks (quartiles), not the mean.

When unsure: plot a histogram of the column first. A roughly symmetric bell shape suggests z-score is reasonable; a long tail on one side suggests IQR is the safer default.

An Outlier Isn't Automatically an Error

This is the part that matters more than the math: a flagged outlier could be a data entry typo (an extra zero turning $50 into $5,000), or it could be a real, legitimate event (a genuinely huge single order). Deleting every flagged row without checking discards real signal along with real noise.

• Investigate before removing. Check a sample of flagged rows manually, is there a pattern (all from one data source, one date, one user)?
• Consider capping instead of deleting (winsorizing) for cases where the outlier is real but you want to reduce its influence on an average, rather than removing the row entirely.
• Document your decision, whatever threshold and action you choose, note it, so the next person analyzing the same data understands what was excluded and why.

Doing This in How To CSV

The Outliers tool detects anomalous values using configurable IQR or z-score thresholds and flags them for review rather than silently deleting anything, giving you the chance to investigate before deciding what to do with each one.