📘

Data Cleaning

Most analysis value is in the cleaning.

v1 data curated

🤖

Install 安装

curl -sSL https://updating.cc/skills/data/data-cleaning/SKILL.md -o ~/.agents/skills/data__data-cleaning/SKILL.md

📄

Raw Source

https://updating.cc/skills/data/data-cleaning/SKILL.md

← Back to profession page

SKILL.md Preview 技能内容预览

---
name: Data Cleaning
description: Most analysis value is in the cleaning.
---
# Data Cleaning

Real-world data is messy: missing values, inconsistent formats, duplicates, units that differ across sources. Skipping cleaning produces confident wrong answers. It is unglamorous but where most of the truth lives.

## Steps
- **Profile first.** Look at distributions, value counts, and types before touching anything. Surprises here (negative ages, future dates) reveal data-entry bugs.
- **Handle missingness explicitly.** Drop, impute, or model — but decide deliberately. "Default to zero" silently corrupts analysis.
- **Standardize units and formats.** Dates in ISO 8601; currency normalized; casing consistent. Inconsistent formats are invisible until they break a join.
- **Deduplicate.** Real duplicates (same record) vs near-duplicates (same entity, slightly different) need different handling.
- **Document every transformation.** A pipeline you can't reproduce is one you can't trust or debug.