Evaluating Calorimeter Reconstruction Algorithms and Calorimeter Designs
Introduction
It is essential to be able to compare different algorithms and different designs in detail to understand their comparative strengths and limitations. A typical difficulty comes when the developer of design/algorithm A presents a study of design A and the developer of design/algorithm B presents a study of design B and the two studies use different standards of comparison.
For example, if A and B are two techniques for identifying calorimeter clusters which were created by gammas. The A study claims a 7% fake rate by dividing the number of clusters incorrectly identified as gammas by the total number of original monte carlo truth gammas in the event. The B study claims a 10% fake rate by dividing by the number of gammas that the algorithm could reasonably have been expected to find, eg, an MC Truth gamma that entirely misses the calorimeter would be excluded from the count. So A looks better than B but if they used the same standard for comparison the relative results might be reversed. We have presented a rather trivial example just to illustrate the point but in real life it can seem difficult to find the time and create the tools to make valid comparisons.
There is an even more important reason for having adequate means for evaluating different designs and algorithms. In the gamma identification example cited above we would really like to be able to simultaneously apply A and B at the same time to the same events and see how A and B do on each gamma. This way we may be able to find a way to combine the strengths of both algorithms and reduce their weaknesses.
In the ClusterID and ClusterAnalysis packages we have tried to develop solutions to these and other related problems to enable designers and developers to focus their attention on improving their products without having to spend a lot of time developing the tools to compare one version of their product with another. For example, you want to be able to ask of a calorimeter design, "What is the optimal segmentation?". But that question may have different answers depending on what you are optimizing, eg, finding gammas, energy resolution, etc. You want a suite of tests that can be easily run to answer these questions.
In the tutorials section we present as an example the code that was used to create the Z mass plots shown in the ClusterID overview and Performance section. In