Interpretation of Significant Differences

Let’s use the following example to illustrate how to interpret the significant differences on the Crosstab page. In the following case, we want to compare people's favourite sportswear brands across different age groups, and in addition we want to validate if the differences observed are significant at 90% confidence level.

With the results in the above table, we can see that 49% 25-34 y.o. consider Nike as their favourite brand and this number is highlighted in green. It indicates we are 90% confident that people aged 25-34 are more likely to consider Nike as their favourite brand when compared to the total sample (33%). Similarly if the number is highlighted in red, it means this number is significantly lower than total.

Explanation of Significance Test

Although this intuitive understanding is essentially correct regarding how to interpret the data, it is not, at a technical level, or a valid way to describe the test. This is because the people in the 25-34 category are included in the total sample. Thus, if we compare those two groups, we would be double-counting - to use the more formal statistical language, we would violate the assumption of independent samples.

The only way that 25-34 y.o. can be different to the total is if they are different to the people that are not 25-34 y.o. The standard solution to this problem is to subtract the data of the 25-34 y.o. from the total, and then perform the significant test. Therefore when we compare the differences between the subgroup and the total sample, we conduct the independent sample test between the subgroup and the total sample minus that subgroup. 

In terms of what test we use, 

• When comparing percentages, we use z test with unpooled variance

• When comparing mean values of scores, we use independent sample t test with equal variance not assured.