Sunday, August 29, 2010

Final Words on Making Great Evidence

In the last two newsletters, we've talked about Credibility and Social Consensus as evidence to answer questions that are asked about claims made in the course of a work discussion. FINALLY we are going to talk about the third and final category of evidence. REAL evidence, that is to say Objective Evidence, means evidence that can be examined and reviewed. Something tangible like data, numbers, testimony, physical articles all qualify as objective evidence. By the way, it doesn’t mean that the evidence itself is objective in content, just that it is tangible (an object) and can be examined by more than one examiner.


Objective evidence is often meant to represent something; that is to serve as an example. Sometimes, a sample will be offered to (like a cracker with cheese on it in a supermarket) and it is offered as objective evidence that this cheese and cracker is representative of the cheese and crackers YOU’LL enjoy if you buy the product. We all know that sometimes this is true and sometimes not. Objective evidence should be subjected to tests in order to determine its validity. Tests for objective evidence are:

• Are the examples representative? OR, were they selected to show certain desirable characteristics?
• From a large enough range? OR, taken from just a few places in the population?
• Are they selected randomly? OR, do they all come from a specific place in the population?
• Is the sample size large enough? OR, were just a few samples taken?
• Are there counterexamples? THAT IS, are there a meaningful number of samples that show opposite characteristics?
• Are the statistics properly prepared? THAT IS, using standard (or logical) statistical techniques?
• Are the objects and testimony authentic? OR, are they fraudulent or modified?

So now we have covered the three kinds of evidence. Let’s talk a little more on how to weigh it. Generally, evidence reported first-hand (by an involved party) is better than second-hand (by a party that was not personally involved, but heard something from someone that was). That doesn’t mean that first-hand evidence is always right or second-hand always wrong, just that GENERALLY, one is better than the other. Also, opinions offered by an expert are GENERALLY better than those of a layperson. The six grades of evidence are as follows (lowest to highest):

1. Assertion (a personal opinion is not very strong by itself)
2. Common Knowledge or Stipulation (meaning there is social consensus regarding an opinion)
3. Lay Opinion (if a reasoned conclusion, meaning there has been some reasoned analysis and there is some credible backing)
4. Expert Opinion or Consensus of Lay Opinion (Expert opinions are backed by the expert’s credibility. A consensus of reasoned conclusions by multiple lay people is as good)
5. An Empirical Study or Consensus of Expert Opinion
6. Consensus of Studies

Now we know how to make a claim, ask the important questions, what evidence to expect to be offered, and how to test it and weight it. It is very important to know that the evidence brought to a discussion is good enough to consider.

Often though, even though the evidence is good evidence (the numbers are right and add up, or the testimony is truthful), it doesn’t actually support the claim. This is one of the biggest problems in these kinds of discussions. Someone says that “sales are down because the economy is bad” and then presents a bunch of evidence that proves the economy is bad, but never actually proves that the economy is why sales are bad.

Next we will talk about the various ways to determine how well the evidence supports the claim, and learn how to make our arguments more robust and “unbreakable”.


Insist on great business results! Go to Pathfinder Communication

Sunday, August 8, 2010

Using Social Consensus as Evidence

Last time we talked about the first of the three kinds of evidence – one’s credibility. I received lots of emails divided into two themes, so I’ll start by answering those for all of you that might have had similar questions.

Yes – credibility is technically a kind of evidence, although I agree that you would have had to take some technical communication courses in order to know that. The reason that it is counted as such is that the purpose of evidence is to support a claim, and an individual’s credibility can often do that. Many times in face to face communication, the speaker’s credibility ALONE is enough to support the claim. The engineer stands up in a meeting and describes why a particular technical approach is superior to another. If I feel his credibility is strong (based on the criteria that I wrote about last time), I may accept his solution without asking for anything else. If it is a matter of vital importance I may ask them how they arrived at their conclusion and, if their process and sources were credible to me, I may accept his conclusion with no other discussion. So credibility alone is used to support some claims and that is why it counts as evidence.

Also, while it is important to assess the credibility of others, it is also important to set the bar at the right height for the topic. The criticality of the outcome usually determines that. In a life or death situation, we need more assurance than if we were making a less serious decision, and so the level of credibility can be relaxed. Interestingly, if a group of listeners are disposed to particular point of view, and the speaker’s point of view supports them, the speaker’s credibility may go unquestioned and rated by the listeners to be very high even though it was never assessed. In other words, if he agrees with us, he must be right. Very dangerous variant on what is called “Confirmation Bias”.

This time I am going to talking about a second kind of evidence- Social Consensus. These are things that we all agree to and require no further support. They are where many conversations go bad, because there are precious few places you can go to find a list of things that we all agree to! So how do you know if a claim is acceptable by social consensus or not?

Well, the first thing we determine is if we have we agreed to it before. If the people in the discussion decided, for instance, that we would allow project managers to fulfill certain responsibilities and they have been doing it without objection then there is some social consensus that those responsibilities are theirs unless we want to question it. In other words, if someone claims that the engineers were doing something that the project managers are supposed to do and the question was “How do we know that the project managers are supposed to do that?”, our evidence could be that there has been a general consensus that it is their job.

Note that this doesn’t mean that we have to continue that way – we can come to a new agreement at any time. All we are saying is that the way we know that the task in question is to be performed by project managers is that there has been a consensus among us that we will handle things that way.

Beware of social consensus! Many things that we believe are universally accepted are NOT. Questioning these is vitally important. Some examples of claims that some may think are generally accepted and are not are:

1 – Democracy is better than non-democracy
2 – People are basically bad
3 – People are basically good
4 – The environment is more important than industry
5 – The current economy is the cause for low sales

You can make your own list by writing down those things that YOU think are generally accepted and asking your friends and coworkers if they agree or not. This is an interesting task because that is the "society" that you are likely trying to understand. If it is like my workplace, you will find that it is very diverse with people from many varying backgrounds. It brings to mind an old saying about "opinions" and their....ummmm... ubiquity.

You will find that most of the things you think are generally true in your business – the things you base your decisions on – are not always as generally accepted as you thought. Some of the most challenging and beneficial discussions are those that bring that to light.

Generally, what constitutes common knowledge are the most simple and non-controversial of facts:
24 hours in a day; seven days in a week; ‘A’ precedes ‘B’ in the alphabet; Los Angeles is in California, 2+2=4. These are all things we generally agree to and would likely never be challenged

Care needs to be taken, though, regarding accepting things that are NOT so “universally known” without some backing. The speaker’s credibility is one of those things; the other will be our topic next time – Objective Evidence


Insist on great business results! Go to Pathfinder Communication