The third week of discussion for our critical thinking group involved thinking about experimental design and statistical analysis.
We read the paper:
Hurlbert, S.H. (1984). Pseudo-replication and the Design of Ecological Field Experiments. Ecological Monographs. 54, p.187-211.
The questions that were raised included:
- What is pseudo-replication and how is it avoided?
- What are the different types of pseudo-replication?
- How do you balance perfect experimental design with feasibility in ecology?
We started off the discussion by summarising the paper, and then delved straight into the questions. The main take home messages from the discussion were as follows:
1. The term pseudo-replication was coined by Hurlbert to refer to “the use of inferential statistics to test for treatment effects with data from experiments where either treatments are not replicated (though samples may be) or replicates are not statistically independent.” making it a classic paper within ecology. We found this definition hard to comprehend straight off the bat so discussed the best explanation of pseudo-replication for a 15 year old at school. Our result can be seen below:
“Imagine you want to find out if temperature of fish tank affects the lifespan of goldfish. You get two fish tanks (each one of these fish tanks we call a replicate), one 10°C and one at 20°C. You put five fish eggs in each tank and wait. When it comes to trying to answer your question, you might find that on average the fish in the colder tank lived longer, great! But, because you only had two fish tanks, really you just found the effect between the two different tanks, and you still don’t know if this is due to the difference in temperatures between them. To fix this, you could have five fish tanks at each of the two temperatures, generating a mean for each temperature and an estimate how much difference there was between the fish tanks at each temperature. With all of this information you can figure out how much of the difference in lifespan was due to temperature versus the different tanks.”
2. Next we discussed the three types of pseudo-replication and their differences;
- Simple pseudo-replication occurs when there is one experimental unit per treatment. Inferential statistics cannot separate variability due to treatment from variability due to experimental units when there is only one measurement per unit.
- Temporal pseudo-replication is similar to simple pseudo-replication but it occurs when experimental units differ enough in time that temporal effects among units are likely, and treatment effects are correlated with temporal effects. Inferential statistics cannot separate variability due to treatment from variability due to experimental units when there is only one measurement per unit.
- Sacrificial pseudo-replication occurs when the means within a treatment are used in an analysis, and these means are tested over the within unit variance. We decided this would be the easiest to account for in statistical analysis than the others because sacrificial pseudo-replication only occurs when carrying out the statistics therefore you just need to redo the statistics instead of the whole experiment unlike simple pseudo-replication.
We thought the paper was clear and concise with the examples referring to specific experiments making it easy to visualise and understand the concept of pseudo-replication. A particularly good figure can be seen below:
3. Finally we talked about how to balance perfect experimental design with feasibility in ecology. We came to the conclusion the “perfect” experimental design is unrealistic within ecology as the environment is constantly changing and there will always be unknowns that cannot be taken into account or measured. If a scientist was set on creating the “perfect” experiment the study would never be put into practice and no knowledge would be gained making the process pointless. Rational thinking needs to be exerted taking into account the facts and variables known as well as the budget and time constraints present to create a balance of these two factors.
Overall a very enthusiastic discussion was had with everyone contributing with their unique views. Looking forward to the next one!
Note from Isla: At the end of the discussion, I also gave a brief intro to hierarchical statistical modelling (mixed models and super briefly Bayesian approaches) as a way to deal with temporal and spatial pseudo replication in many experimental designs.