How to interpret this figure...

Cloud fraction model intercomparisons

Mean cloud fraction

Panel (a) shows mean cloud fraction versus height for the observations and the models. The grey region shows the observed mean cloud fraction, calculated from the radar and lidar data on the vertical grid of each model and subsequently regrided on to a regular 1-km grid to allow comparison between the various models. The width of the grey region arises due to the different availabilities of the various models, resulting in the observations being averaged over different time periods. The different vertical grids also causes a slight difference. Visit the level 3 cloud fraction comparisons for the individual models to see a more direct comparison with a particular model. Above a certain rainfall rate there is a danger that, due to strong attenuation, the radar may underestimate cloud fraction. Therefore this mean does not include situations with a surface rainrate above 8 mm/hr in the case of 35-GHz radars, and 2 mm/hr in the case of 94-GHz radars.

The coloured lines show the corresponding mean cloud fraction taken from the models when observational data were available. Periods when the rain rate exceeded the thresholds given above have been excluded. The dashed lines show the values taken directly from the model, but the radar can have problems in detecting tenuous ice clouds so for a fairer comparison, the solid lines shows the mean of the model values after filtering to remove ice clouds estimated to be undetectable by the radar, based on the known radar sensitivity and an estimate of the variation of radar reflectivity on a scale equivalent to the horizontal gridbox size of the model (Hogan and Illingworth 2003, J. Atmos. Sci., 60, 756-767). There is considerable uncertainty in this procedure, so where the solid and dashed lines diverge significantly, it is difficult to make a confident comparison with the observations. Usually this tends to occur only above 7-8 km.

Skill scores

Panels (b)-(g) depict two skill scores that present a measure of how well the individual cloud features were predicted, the Equitable threat score and Yule's Q. Both have the property that a perfect forecast scores 1 while a random forecast scores 0. These two skill scores have been chosen because they are known to be relatively insensitive to the frequency of occurrence of the property being assessed (unlike scores such as hit rate or false alarm rate).

The scores are calculated as follows. Firstly a threshold cloud fraction is chosen. A contingency table is defined, such that A is the number of times that cloud fraction exceeded the threshold in both the model and the observations, B is the number of times that cloud fraction exceeded the threshold in the model but not the observations, C is the number of times that cloud fraction exceeded the threshold in the observations but not the model and D is the number of times that cloud fraction exceeded the threshold in neither the model nor the observations. The scores are defined by:

• Equitable threat score ETS = (A-E)/(A+B+C-E), where E is the number of hits that occurred by chance, given by E=(A+B)*(A+C)/(A+B+C+D). This score is an improvement on the Threat score TS = A/(A+B+C), which does not have the property of a score of 0 corresponding to a random forecast.
• Yule's Q skill score Q = (A*D-B*C)/(A*D+B*C). It is closely related to the odds ratio but conveniently ranges between -1 and 1.

Panels (b) and (c) show the two skill scores (for data at all heights) versus the cloud fraction threshold. Note that some of the variation with threshold occurs not because of any change in skill at forecasting different values of cloud fraction, but because of a change in the frequency of occurrence of cloud fractions above different thresholds.

Panels (d) and (e) show the skill scores for a threshold cloud fraction of 0.05 as a function of height above the ground.

Panels (f) and (g) show the skill scores for a threshold cloud fraction of 0.05 as a function of month through the year. Note that in some months there may be either low cloud fraction or intermittent performance by instruments or model, resulting in rather more noisy skill scores.