Questionnaire based studies.

A lot of student projects use questionnaires, either as the only method of data collection or alongside other types of data, such as a questionnaire on physical activity or diet habits along with measurements of blood cholesterol.

It is important that your questionnaire produces data that you can analyze in a meaningful way.

1.Types of Studies

1.1    Observational studies

An observational study is one where we try to measure/ estimate something in a ‘real life’ setting, without influencing the thing we are seeking to measure.

For example,

• How much fruit and veg do people eat
• Hygiene practices in domestic kitchens
• Knowledge about GM crops
• Levels of stress in students

Most of these are done at one time point only, but they can be repeated at different time points. For example you might measure stress in students before and after the exam period (note this is not an INTERVENTION study as you haven’t done anything in your study to modify stress, just measured what is already happening).

1.2    Intervention studies

Rather than an observational study you can use a Questionnaire before and after deliberately attempting to change the thing we are measuring, via an  ‘intervention’ or treatment

E.g. Stress questionnaire before and after meditation intervention, physical activity questionnaire before and after putting up signs to promote activity in the workplace.

In these studies you use exactly the same questionnaire twice in the same population. This can give paired/ repeated measures data IF you are able to match up responses before and after for individuals.

 

2.Analyzing your data

 

2.1    Comparing with existing data

You can just make observations on a single group/ population, but this is then very limited in what you can conclude. To discuss what your results mean you need some sort of comparison with other information. This gives EXTERNAL VALIDITY to your data.

2.1.1     Comparing with formal guidelines and/ or official statistics

Formal guidelines / levels

• g. Government guidelines of 5 a day fruit and veg – does your population meet the guidelines?

National/ local official statistics from government/ council etc.

• g. Physical activity levels in the population- is your population better or worse than published stats?

2.1.2     Comparing with published research data

There may be studies that have looked at similar data and published information you can compare with.

2.1.3     Making sure you will be able to compare your data

For all of these types of published/ official data it is important that you collect your own data in a format that can be compared exactly, for example using exactly the same age categories, ethnicity categories or definitions of physical activity. Where possible get hold of the EXACT questions used.

2.2    Comparing more than one ‘group’

As well as comparing with existing data, for many projects comparing at least 2 different ‘groups’ will give you much more to discuss- for example

• Male/female,
• young/old
• biology students and maths students
• different ethnic groups

 

For your different groups you need:

• Sensible reason why the groups might be different- based on previous literature etc.
• Clear group criteria so you can assign people to specific groups- again need evidence base from literature for some of these, e.g. if looking at different ages does the literature suggest specific ages that have different values ?

Note that you can analyze the same data set for more than one set of groups

For example you might have collected information on how many fruit and veg are eaten by 100 students- you could then analyze males and females but also analyze different faculties, students who live with parents and in halls, overseas and home students  etc, as long as

• You collected the information to assign them into the different groups as part of your questionnaire.
• You have sufficient subjects in each group for meaningful comparisons.

This approach can work well where a group of project students design a questionnaire together and work together to get lots of respondents, then each analyze a specific set of groups

e.g. you could collect data about smoking habits in students

• project student 1 could look at male and female differences
• project student 2 could look at differences between students on health courses and students from other courses
• project student 3 could look at differences between ethnic groups

In each case there is a lot of literature about differences in these groups so a lot for the students to discuss. By working together they will get better subject numbers and are far more likely to get meaningful statistical analysis.

2.3    Statistical analysis

You can present all your questionnaire results as mean/ mode/ median values etc. and talk about differences between your data and other data or between groups in your own data. However, unless you can carry out statistical analyses it is difficult to conclude if your findings are meaningful.  It is very important that you consider statistical analysis at the design stage of your questionnaire so you collect information in a format that can be analyzed.

3.DATA TYPES AND ANALYSIS

Before we look at question design we need to remember some key information about data analysis

3.1    REVISION OF SOME OF THE BASICS

P VALUES

P= Probability

• Value from 0 to 1.00 (i.e. 0 to 100%)
• usually take P<0.05 as significant

 

PARAMETRIC AND NON-PARAMETRIC DATA

PARAMETRIC- MUST COMPLY WITH ALL OF	NON-PARAMETRIC- ANY ONE OR MORE OF
normal distribution	skewed  distribution
equal variance	unequal variance
independent observations	dependent observations

 

ONE-TAILED OR TWO TAILED

One tailed if you are sure the difference can only be in one direction Two tailed if difference could be in either direction (e.g. the value in males could be higher or lower than in females).

 

PAIRED/ REPEATED MEASURES

Your data is PAIRED if you have more than one measurement made on the same subjects/ same sample.

For example if you measure blood pressure in a group of people before and after you give them a drug, or if you take a single batch of cells, split it in half and give a different treatment to each half. If you have paired date you use PAIRED t-test or REPEATED measures ANOVA