Hmm, I think that maybe we can do something like this, accumulate lots of data to help people minimize visits to the doctors and the lab test. Need to study this further.
Thanks To Brad Delong , from here:
Cosma Shalizi on how we are not as smart as the simple linear models our computers can estimate:
Clinical and Actuarial Judgment Compared: For something like fifty years now, psychologists have been studying the question of “clinical versus actuarial judgment”…. Say you’re interested in diagnosing heart diseases from electrocardiograms. Normally we have clinicians, i.e., expert doctors, look at a chart…. Alternately, we could ask the experts what features they look at, when making their prognosis, and then fit a statistical model to that data, trying to predict the outcome or classification based on those features…. This is the actuarial approach, since it’s just based on averages — “of patients with features x, y and z, q percent have a serious heart condition”.
The rather surprising, and completely consistent, result of these studies is that there are no known cases where clinicians reliably out-perform actuarial methods, even when the statistical models are just linear classification rules…. In many areas, statistical classifiers significantly out-perform human experts. They even out-perform experts who have access to the statistical results, apparently because the experts place too much weight on their own judgment…. [H]uman experts are… no better than simple statistical models.
On the other hand, there is another body of experimental work, admittedly more recent, on “simple heuristics that make us smart”, which seems to show that people are often very good judges, under natural conditions. That is to say, we’re very good at solving the problems we tend to actually encounter, presented in the way we encounter them. The heuristics we use to solve those problems may not be generally applicable, but they are adapted to our environments, and, in those environments, are fast, simple and effective.
I have a bit of difficulty reconciling these two pictures in my mind. I can think of three resolutions.
- The “clinicial versus actuarial” results… do not reflect the “natural” conditions of clinical judgment…. What one really wants is a representative sample of actual cases, comparing the normal judgment of clinicians to that of the statistical models. This may have been done; I don’t know.
- The “fast and frugal heuristics” results are… irrelevant…. [A]daptive mechanisms [that] let us figure out good heuristics in everyday life don’t apply in the situations where we rely on clinical expertise…. [S]omething… about the conditions of clinicial judgment… render our normal cognitive mechanisms ineffective there.
- Clinicial judgment is a “fast and frugal heuristic”, with emphasis on the fast and frugal…. [C]linicians are… as accurate as one can get, using only a reasonable amount of information and a reasonable amount of time, while still using the human brain, which is not a computing platform well-suited to floating-point operations…
I am unable to judge between these.
