What data science actually is

A day in the life: a question that needs data

Imagine you run a small bubble-tea shop. Last week you added a new drink — taro milk tea — and it felt like a hit: the staff were busy, the queue looked long, and you went home tired and happy. On Monday you tell a friend, “The taro milk tea is really popular.” Your friend, who is a careful sort of person, asks a deceptively simple question: “How do you know?”

That question is where data science begins. Right now you have a feeling, and feelings are easy to fool. Maybe last week was simply payday week, when everyone buys more of everything. Maybe the queue looked long because one machine broke, not because demand rose. Maybe taro milk tea sold well but stole sales from your bestselling brown-sugar tea, so total revenue didn’t actually move. A hunch can’t tell these stories apart. Data can — if you ask it the right way.

Turning a vague feeling into a question you can actually answer is the first and most underrated skill in the field. Compare “Is taro popular?” (too fuzzy to answer) with “Did adding taro milk tea raise our average daily revenue compared with the four weeks before it launched?” The second version names what to measure, over what period, against what comparison. This whole track — and this guide — is about the disciplined, honest way to go from a question like that to a decision you can defend.

Data science vs statistics vs machine learning, in plain words

You will hear three words used almost interchangeably — statistics, data science, and machine learning — and beginners often worry they are missing some sharp boundary between them. The honest truth is that the boundaries are fuzzy and even experts argue about them. But each word does have a center of gravity, and a simple picture helps.

Statistics is the oldest of the three: it is the mathematical science of learning from data in the presence of uncertainty. Statistics gives us the careful machinery for asking “Given the wiggly, noisy numbers I collected, what can I responsibly conclude — and how wrong might I be?” It is the part that keeps everyone honest, and it is the heart of this whole track.

Data science is broader and more practical. It is the whole craft of turning raw data into useful knowledge: finding and collecting data, cleaning the mess, exploring it, analyzing it (using statistics), and — crucially — communicating the result to people who will act on it. A data scientist needs a bit of statistics, a bit of programming, and a bit of knowledge about the actual problem (tea sales, hospital wait times, ad clicks). Think of statistics as a core toolbox and data science as the full job that uses that toolbox in the real world.

Machine learning is a particular family of techniques, mostly aimed at one job: making predictions by finding patterns in data automatically. A spam filter that learns which emails are junk, or a model that guesses which customers will churn, is doing machine learning. It overlaps heavily with statistics (many methods are shared) and lives inside the larger area called artificial intelligence. For now, just hold this picture: machine learning is one powerful tool a data scientist sometimes reaches for — not the whole job, and not the focus of this track.

The workflow loop: question → collect → clean → explore → model → decide

Almost every real project follows the same arc, whether it takes an afternoon or six months. People give it different names — the data-science lifecycle is one — but the steps are remarkably stable. Here is the loop, applied to our taro milk tea question.

1. Ask a sharp question. Turn “is taro popular?” into something measurable: “Did launching taro milk tea raise average daily revenue, versus the four weeks before?” Decide up front what would count as a yes.
2. Collect the data. Pull the right records — daily sales, item-by-item receipts, the launch date, maybe the weather. Decide what counts as one row and write down where each number came from.
3. Clean it. Fix typos, drop duplicate receipts, handle missing prices, make dates consistent. This unglamorous step is where most of the time goes — and where most mistakes hide.
4. Explore it. Plot the numbers before you trust them. This is exploratory data analysis (EDA): look at the shape, the spread, the surprises. Eyes catch things formulas miss.
5. Model it. Use a method that actually answers the question — here, comparing two periods while accounting for noise. The “model” can be as simple as a careful average or as fancy as a regression.
6. Decide and communicate. Translate the result into an action (“keep taro, drop the worst-selling tea”) and tell people honestly how sure you are — including what could still be wrong.

Two honest notes about this loop. First, it is iterative: you will rarely walk it once and be done. Exploring the data often reveals that your question was wrong, sending you back to the start. Second, the glamorous-sounding steps (modeling, machine learning) are usually the smallest part of the work. Surveys of practitioners repeatedly find that collecting and cleaning data eats well over half the time. If you learn to love that part instead of resenting it, you will be ahead of most people.

Kinds of data: rows, columns, and variable types

Before we can analyze anything, we need a shared picture of what “data” looks like. Most of the time, it is a table — like a spreadsheet. Each row is one thing you observed (one order, one customer, one day), called an observation. Each column is one property you recorded about those things (the drink name, the price, the date), called a variable. A single cell holds one value: the price of one specific order.

When a table follows one simple rule — one row per observation, one column per variable, one value per cell — we call it tidy data. Tidy data is not fussiness for its own sake; it is the shape that every later tool (charts, summaries, models) quietly expects. Messy data is the number-one reason analyses stall, so it is worth recognizing the tidy shape early.

Now the crucial idea: not all columns are the same kind. Variable types split into two big families. A categorical variable records which group something belongs to — drink name (taro, brown-sugar, green tea) or size (small, medium, large). Some categories have a natural order (small < medium < large — these are called ordinal), while others have none (the drink names — these are called nominal). A numerical variable records a quantity you can do arithmetic on — price in dollars, or number of items in an order. Numbers come in two flavors too: discrete (whole counts, like 3 items) and continuous (any value on a scale, like a weight of 412.7 grams).

import pandas as pd

# Each row is one order; each column is one variable.
orders = pd.read_csv('orders.csv')

orders.head()                   # peek at the first few rows
orders.describe()               # quick numeric summary: mean, min, max...
orders['drink'].value_counts()  # count orders per category

The two jobs of statistics: describe vs infer

Statistics does two distinct jobs, and keeping them apart will save you endless confusion. The first job is to describe the data you actually have. The second is to infer something about a larger world you cannot fully see. Beginners blur these together; professionals are obsessive about which one they are doing.

Descriptive statistics simply summarizes the numbers in front of you. If 500 customers bought drinks today, the average price they paid, the most common drink, the highest and lowest bills — these are summary statistics, and they make no claim beyond today’s 500 customers. The most familiar one is the mean (the everyday “average”): add up all the values and divide by how many there are. (When a few huge bills would distort the average, the median — the middle value — is often a fairer summary.)

The sample mean in symbols: x-bar is the sum of all n values divided by n. We introduced it in words first — a formula is just shorthand for an idea you already understand.

Here is the catch that motivates the second job. You almost never have data on everyone you care about. You care about all your customers, today and in the months ahead — that entire group is the population. But you only ever measure a slice of it: the people who happened to come in, the days you happened to record. That slice is a sample. The whole adventure of statistics is using the sample you have to say something trustworthy about the population you don’t.

Inferential statistics — also called statistical inference — is exactly that leap, made carefully. Because the sample is only a slice, any number you compute from it (like today’s average bill) would come out a bit differently if you had sampled different customers. That wobble is real and measurable; we call its typical size the standard error. A wonderful fact, which the next two guides unpack, is that this wobble shrinks as your sample grows — roughly in proportion to one over the square root of the sample size.

A preview, not a formula to memorize: the typical wobble of a sample average is roughly the spread of the data (σ) divided by the square root of the sample size (n). Quadruple your sample, and the wobble only halves.

What “good” looks like: honesty, uncertainty, reproducibility

Most of the difference between good data work and bad data work is not technical skill — it is intellectual honesty. The tools are easy to run; the discipline to report what they really say is rare. Three habits separate trustworthy analysts from the rest, and this whole track will keep returning to them.

First, always carry your uncertainty with your answer. A single number like “revenue rose 6%” hides how shaky it might be. That is why we report ranges. A confidence interval is the standard way to say “my best estimate is 6%, and the true value is plausibly somewhere between 1% and 11%.” But beware the most common misreading in all of statistics: a 95% confidence interval does NOT mean “there’s a 95% chance the true value is in this particular range.” The 95% describes the procedure — if you repeated the whole sampling-and-computing process many times, about 95% of the intervals it produces would contain the true value. The interval in front of you either contains it or it doesn’t. Guide 4 makes this precise; for now, just respect that the “95%” is a property of the method, not a bet on this one interval.

Second, never confuse a pattern with a cause, and never confuse “statistically significant” with “important.” That ice cream sales and drownings rise together does not mean ice cream drowns people — a hot summer drives both. This is the correlation-is-not-causation trap, and an entire track is devoted to escaping it. Likewise, with a big enough sample you can detect a revenue change so tiny it would never pay back the cost of the new drink: real, but practically worthless. Statistical significance answers “is this probably not just noise?”; it does not answer “is this big enough to care about?” Always ask both.

Third, make your work reproducible. If you cannot rerun your analysis next month and get the same answer, you cannot really trust it — and neither can anyone else. Reproducibility means writing down every step as code rather than clicking around by hand, keeping the raw data untouched, and recording the choices you made. It feels like extra work at first, and it is the single biggest favor you can do your future self.

What’s ahead in this track

You now have the map. This track — Probability & Inference — builds the core reasoning that every later track leans on. Here is the road, each guide a single big idea.

1. Randomness, variables & distributions — the grammar of uncertainty: probability, random variables, and the handful of distribution shapes that describe most of the world.
2. Sampling & the central limit theorem — why a few thousand well-chosen people can speak for millions, and why averages become predictably bell-shaped.
3. Estimation & confidence intervals — putting an honest number, with a range, on what you don’t know.
4. Hypothesis tests & p-values — the ritual everyone uses and most people misread, finally demystified.
5. Bayesian thinking — the other way to do statistics: start with a belief, update it with evidence, read the answer as a usable probability.

Beyond this track lie the places these ideas go to work. Experiment design teaches the A/B test — how companies actually learn what works by randomizing. Regression, built on ordinary least squares, measures relationships and makes predictions. Causal inference confronts the hardest question of all — does X truly cause Y? And the wrangling, visualization, and data-stack tracks cover the practical craft of getting messy data into shape, seeing it clearly, and running it all reliably, right up to machine learning in production.

You don’t need any of it memorized today. You only need the mindset you already practiced in this guide: ask a sharp question, respect the gap between your sample and the world, and stay honest about how sure you are. Carry that, and the rest is just learning the tools, one clear idea at a time. Turn the page when you’re ready.