Introduction to Regression Analysis: Basics and Applications

Every environmental scientist eventually faces a core question: how does one variable influence another? Whether it’s understanding how rising CO₂ levels affect global temperatures or how deforestation rates relate to local rainfall, answering these questions requires a reliable statistical method. That method is regression analysis – one of the most widely used tools in research for modelling relationships between variables and making predictions based on data.

In this post, we’ll break down what regression actually means, trace its fascinating historical origins, and explore the types of regression models – with a close focus on simple linear regression – that form the backbone of environmental science research.

What is regression? 🔗

At its core, regression analysis is a statistical technique used to estimate the relationship between a dependent variable (also called the response variable) and one or more independent variables (also called explanatory or predictor variables). The dependent variable is what you’re trying to predict or explain, while the independent variable is the factor you believe influences it.

For example, suppose a researcher wants to know how stream temperature changes with elevation. Here, elevation is the independent variable, and stream temperature is the dependent variable. Regression analysis helps quantify this relationship mathematically, producing an equation that describes how one variable changes as the other changes.

Regression does more than just show whether a relationship exists. It serves three main purposes. First, it identifies relationships between variables – whether they are positive, negative, or non-existent. Second, it quantifies the strength of those relationships, telling researchers exactly how much the dependent variable changes for each unit change in the independent variable. Third, and most importantly, it enables prediction – allowing researchers to estimate the value of one variable based on the known value of another.

In environmental research, regression is used extensively. Climate scientists model the connection between greenhouse gas concentrations and temperature anomalies. Hydrologists study how rainfall patterns relate to river flow rates for flood prediction. Ecologists examine how habitat fragmentation affects species diversity to inform conservation planning. Pollution researchers use regression to estimate how land use and emissions relate to air quality in urban areas.

The origin of regression 🔗

The term “regression” in statistics has a surprisingly specific origin – and it wasn’t initially about fitting lines to data at all. It traces back to the work of Sir Francis Galton, a 19th-century British scientist and cousin of Charles Darwin.

Galton’s study of hereditary height 🔗

In the 1880s, Galton was deeply interested in heredity – specifically, how traits like height pass from parents to children. He collected height data from 930 adult children and their parents, plotting each child’s height against the average height of their two parents (what he called the “mid-parent height”).

What Galton expected was straightforward: tall parents should have equally tall children, and short parents should have equally short children. If that were the case, the data points would follow a steep line where child height perfectly matched parent height. But that’s not what he found.

Regression toward mediocrity 🔗

Instead, Galton observed that while tall parents did tend to have taller-than-average children, those children were not quite as tall as their parents. Similarly, short parents had children who were taller than themselves, though still shorter than average. The offspring heights tended to drift back toward the population average. Galton described this as children being “more mediocre” than their parents – smaller if the parents were large, larger if the parents were small.

He called this phenomenon “regression toward mediocrity” – what we now know as regression to the mean. The word “regress” literally means to go back or revert, and Galton used it to describe this tendency of extreme values to move closer to the average upon repeated measurement.

From biology to statistics 🔗

While Galton’s initial interest was biological, the method he used to study this phenomenon – fitting a line through data points to describe the relationship between two variables – became the foundation of modern regression analysis. Galton’s work on inherited characteristics led to the initial conceptualisation of linear regression, while subsequent work by Karl Pearson expanded it into more general techniques including multiple regression and the product-moment correlation coefficient.

What began as an observation about human height has since become an indispensable tool across virtually every scientific discipline – from economics and medicine to environmental science and engineering.

Understanding regression to the mean in practice 🔗

Regression to the mean is not just a historical curiosity. It has real implications for how we interpret data today, including in environmental science.

Consider a practical example: a city installs speed cameras at road junctions where an unusually high number of accidents have occurred. After installation, accident numbers drop. Does this prove the cameras worked? Not necessarily. Those junctions were selected precisely because they had extreme accident counts. Statistically, those counts were likely to decrease on their own – with or without cameras – simply because extreme values naturally tend to be followed by less extreme ones.

In environmental monitoring, the same principle applies. If a particularly polluted site is chosen for remediation, pollution levels may decrease over time partly due to regression to the mean rather than the intervention alone. Researchers must account for this effect by using control groups and baseline measurements to distinguish between genuine improvements and statistical artefacts.

Types of regression models 🔗

Regression analysis isn’t a single technique – it encompasses a family of methods, each suited to different kinds of data and research questions. Here’s an overview of the main types.

Simple linear regression 🔗

This is the most fundamental form of regression. It models the relationship between one independent variable and one dependent variable using a straight line. The assumption is that the relationship between the two variables is approximately linear.

The equation for simple linear regression is:

ŷ = b₀ + b₁x

Here, ŷ (pronounced “y-hat”) is the predicted value of the dependent variable. b₀ is the y-intercept – the predicted value of y when x equals zero. b₁ is the slope – the amount by which y changes for each one-unit increase in x. And x is the independent variable.

For instance, an environmental researcher might use simple linear regression to study whether temperature predicts plant height. If the resulting equation is ŷ = 2.1 + 0.35x, this means that when temperature is 0, the predicted plant height is 2.1 units, and for every one-degree increase in temperature, plant height increases by 0.35 units.

How the line is fitted: the least squares method 🔗

The regression line isn’t drawn arbitrarily. It’s determined using a method called ordinary least squares (OLS), which finds the line that minimises the total squared distance between each data point and the line itself. These distances are called residuals – the difference between the actual observed value and the value predicted by the line.

By squaring each residual and summing them up, the least squares method ensures that the line sits as close as possible to all data points collectively. This produces the “best fit” line for the given data.

Key components of regression output 🔗

When you run a regression analysis, the output gives you several important pieces of information:

Slope (b₁): This tells you the direction and rate of the relationship. A positive slope means both variables increase together. A negative slope means one increases as the other decreases. The estimated change in the dependent variable for each one-unit increase in the independent variable is what the slope represents.

Intercept (b₀): This is the value of y when x is zero. In some cases, this value is meaningful (e.g., a baseline pollution level when a source is absent). In other cases, it may not have a practical interpretation but is still necessary for the equation.

R² (coefficient of determination): This measures how much of the variation in the dependent variable is explained by the independent variable. An R² of 0.85 means 85% of the variability in y can be accounted for by x. The remaining 15% is due to other factors not included in the model.

Residuals: These are the differences between observed and predicted values. Examining residuals helps researchers assess whether the linear model is appropriate for their data. If residuals show a pattern (e.g., they fan out or curve), it may indicate that a linear model is not the best choice.

Multiple linear regression 🔗

When a single predictor variable isn’t enough, multiple linear regression extends the model to include two or more independent variables. The equation becomes:

ŷ = b₀ + b₁x₁ + b₂x₂ + … + bₙxₙ

In environmental science, this is extremely common. A researcher studying plant growth might include temperature, soil moisture, and sunlight as independent variables simultaneously. Multiple regression allows them to determine which factors have the strongest influence and how they interact.

For example, a study modelling water quality might include land use changes, precipitation, and industrial discharge as predictors to explain variation in downstream pollutant concentrations. This gives a more complete picture than examining any one factor in isolation.

Non-linear regression 🔗

Not all relationships in nature follow a straight line. Non-linear regression models account for curved or more complex relationships between variables. These can take many forms – polynomial, exponential, or logistic – depending on the underlying biology or physics.

For instance, the relationship between temperature and the metabolic rate of certain organisms often follows an exponential curve rather than a straight line. In such cases, fitting a linear model would produce misleading results, and non-linear regression provides a more accurate representation.

Simple linear regression in environmental science: practical applications 🔗

Simple linear regression is especially valuable in environmental science because many fundamental relationships are approximately linear, at least within the range of observed data. Here are some key applications:

Climate and temperature modelling: Researchers use simple regression to examine how variables like elevation, latitude, or CO₂ concentration relate to temperature. The U.S. Environmental Protection Agency uses regression analysis in stream ecosystem studies to model environmental conditions as a function of geographic and land-use variables.

Air pollution studies: Regression helps establish the relationship between emission sources and measured pollutant concentrations, enabling agencies to develop better pollution control strategies.

Water resource management: By modelling the relationship between rainfall and river flow, hydrologists can better predict floods and plan water supply systems.

Ecological research: Conservation biologists use regression to understand how changes in habitat area relate to changes in species abundance, helping to prioritise areas for protection.

Assumptions and limitations of linear regression 🔗

Linear regression is powerful, but it comes with assumptions that must be met for results to be valid. Understanding these is essential for any researcher applying the technique.

Linearity: The relationship between variables must be approximately linear. If the true relationship is curved, a linear model will produce poor predictions.

Independence: Observations should be independent of one another. Samples collected from the same site or repeated measurements of the same subject may violate this assumption, leading to inflated confidence in results.

Normality of residuals: The errors (residuals) should follow a roughly normal distribution. Researchers can check this using tools like Q-Q plots – if residuals deviate substantially from a straight line on the plot, the normality assumption may be violated.

Constant variance (homoscedasticity): The spread of residuals should remain consistent across all levels of the predicted values. If residuals fan out (a pattern sometimes called a “trumpet shape”), the model’s predictions will be less reliable at certain ranges.

Correlation is not causation: A regression model can show that two variables are strongly related, but it cannot by itself prove that one causes the other. A strong correlation between deforestation and declining species counts is suggestive, but establishing causation requires additional evidence from controlled studies and ecological theory.

Getting started with regression: a simple framework 🔗

If you’re new to regression, here’s a straightforward process to follow:

Define your research question. Be clear about what relationship you’re investigating. What is your dependent variable, and what do you think influences it?

Collect and organise your data. Ensure your data is reliable, with both variables measured on a continuous scale.

Visualise the data. Create a scatter plot to get an initial sense of whether a linear relationship exists.

Fit the model. Use statistical software to calculate the regression equation and assess key metrics like the slope, intercept, and R².

Check assumptions. Examine residual plots and Q-Q plots to verify that your model meets the necessary assumptions.

Interpret and report. Explain what the slope and intercept mean in the context of your research question, note the strength of the relationship, and acknowledge limitations.

Why regression matters for environmental research 🔗

Regression analysis provides environmental scientists with a rigorous, quantitative framework for understanding the natural world. From predicting how ecosystems respond to climate change, to evaluating the effectiveness of pollution controls, to modelling species distribution under different habitat scenarios – regression turns raw data into actionable knowledge.

The technique that Francis Galton developed while studying the heights of Victorian families has evolved into one of the most essential tools in modern science. Its simplicity makes it accessible; its mathematical foundation makes it reliable. And in a field like environmental science – where decisions based on data can shape policy and protect ecosystems – having the right analytical tools isn’t just helpful. It’s essential.

What do you think? How might you use regression analysis to investigate an environmental question in your own area – such as the relationship between urbanisation and local air quality, or between rainfall and groundwater levels? And could regression to the mean be misleading your interpretation of any trends you’ve noticed?

References

1. https://www.epa.gov/caddis/basic-analyses-1
2. https://www.sciencedirect.com/science/article/pii/S0160412024000163
3. https://select-statistics.co.uk/blog/regression-to-the-mean-as-relevant-today-as-it-was-in-the-1900s/
4. https://rss.onlinelibrary.wiley.com/doi/full/10.1111/j.1740-9713.2011.00509.x
5. https://www.tandfonline.com/doi/full/10.1080/10691898.2001.11910537
6. https://environmentalcomputing.net/statistics/linear-models/linear-regression/
7. https://online.stat.psu.edu/stat200/book/export/html/244
8. https://www.numberanalytics.com/blog/regression-analysis-environmental-data