Well, just pick one and prepare for it. Correct? Buzz! Wrong!

Judgement day: a compendium of misfortunes

Lets start with an old look at this problem. The Book of Revelation was written by the apostle John, and is the last book in the Bible. A fitting place, just like The Last Battle is the final book in the Narnia series written by C. S. Lewis which brings the Narnia series to an end. (I sure hope they do a movie on this. But it will be chilling if they do.)

The Bible starts with “In the beginning…” in Genesis and ends with the total annihilation of earth, and then the presenting of the “new heavens and new earth” of a very different, other-dimension type of place. I think of this as a product recall, scraping the old, giving a new, better replacement.

To stop digressing, Revelation contains a lot of descriptions regarding the end of the world, and not at one time or place. More like a continual bombardment of misfortune. There are wars, plagues, famines, things falling from heaven into the seas (asteroid? volcano? toxic waste? a piece of the falling sky?) and all sorts of maladies resulting from that.

As a child, these stories filled me with terror. As an adult, I see bits and pieces of all these maladies in our present world. And ask anyone who is so unfortunate as to be living on location where one of these maladies is playing out; they would say, it is the end of the world.

Let’s spin the wheel of misfortune—it’s shake and bake time

One of my favorite comic movie about survival is Packin’ It In (TV 1983). The movie is survival spoof on escaping civilization by running to the wilds to live. Everyone was prepared for some form of disaster. The one family even had a cement bunker and weapons, ready for nuclear war. Other families had their specialties. And what happened? It rained so hard that they had mud slides that wiped out all their cabins and filled the bunker with mud. Everyone lost everthing. No one was prepared.

I loved the movie 2012–the end of the world as a massive geologic restructuring of everything, a destructive rearrangement of everything. We see this everyday. Businesses call this downsizing. Although the geologic truth is stretched, it makes a great movie.

By the way, why would people think the end of the world would occur in 2012. The civilization that put that calendar together ended hundreds of years ago, far ahead of schedule.

Let’s spin the wheel of misfortune—right off its axis

Let’s use risk assessment and mitigation techniques: risk assessment, risk mitigation. What will this enable us to do? To devise multiple scenarios, determine the consequences of each, and assist the probability that this scenario may come about. Or more likely, the probability that parts of this scenario will happen.

This information will give some common areas of risk that result from any number of scenarios. For these, the risks need to be studied and mitigated. This will provide us with a set of common problems and the solutions to them. We may not be prepared for a specific scenario; however, we are prepared for the more common consequences from a number of likely scenarios.

An example

If the scenario is a severe plague or flu, then our infrastructure may be brought down. A one-year supply of food would help to sit this one out. Or if there is a nuclear EMP attack to the US as described in One Second After, there is no fallout but a total infrastructure collapse. And a one-year supply of food would truly save your family’s lives. And if a small asteroid or comet hits earth, a one year supply of food might be stretched out further until the dust settles.

This is just an example to show that the specific action of “one-year supply of food” provides survival and benefits for a number of scenarios. There are other important actions that help in multiple scenarios: having a few weapons, living in a far away place, having silver and gold coins for barter, learning how to garden or other skill for living without modern infrastructure, books, medicines, and many more.

And there are a number of activities, such as gaining skills for living off the land, that help us in our present situation, and not just in the by-and-by.

Enemies of the wheel of misfortune—risk assessment and mitigation

Risk assessment and mitigation will provide us with the greatest bang for our buck, the best return on our valuable time. With these tools we can determine which things have the greatest probability and concentrate on those.

Shamelessly selling my own product

The multiple-criteria, multiple-option program will prove helpful in sorting out the best and worst candidates for consideration. The program will assist in culling out the unlikely and unimportant.

This program is free with the instructions on how to use the program in an earlier blog. The link to the downloadable code is Google Open Source Code: MCMO Decision Analysis Program. Help yourself.

http://code.google.com/p/multiple-criteria-multiple-option-decision-analysis/downloads/list

This is a final and full version, albeit, a work in progress. This is not a trial version that frustrates you with partial functionality that ends up wasting your time. It does not have popups telling you that if you pay some extortion fee you can get the “real thing.”

Yes, there are a bunch of features that would sure be nice to have, like saving files and opening them. Next version. Wanting to get this program into the public’s hands, some important functions were relegated to a later time and place. Since I also use this program, I also want those extra features, too. They are coming down the pike.

A look at the finished product

First, we will select three options and all the criteria from the previous blog for the demonstration.

Options:

• Take a cruise
• Take a closer vacation to the beach or mountains or lake
• Visit some relatives

Criteria:

• Which will cost me the least (or put the most money in my pocket)?
• Which is something that more people in the family would like rather than only one or two individuals?
• Which would take the least time to get to and from, to avoid wasted travel time that takes away from our fun time?
• Which is the easiest to do, at hand?

When entered in the main form, the form looks like this:

First the criteria must be ranked. Click on the Rank Criteria button to get to the voting form for all criteria.

How does the left item compare with the right item? Simply click the equal sign if they are about the same. Or click an arrow to show that one is favored over the other. The double arrow means the criterion is much more favorable than the one on the other side.

When every possible pair of criteria have been compared, the Finish button is enabled and a message that you are done appears as follows:

This takes us back to the main form. The two additional buttons enabled are to display the results of ranking the criteria, or to proceed further with ranking the options against the criteria.

The following shows the displaying of the ranked criteria.

Note that the most important criterion is what is the most fun to the most family members. In second place is the time required to get to and from the vacation destination.

Proceed to ranking the options. Compare every pair of options in light of the first criterion. Then compare every pair against the second criterion, and the third, and so on.

Once all the criteria have been used as the measure by which to compare all the pairs of options, the form indicates this with a message and the Finish button enabled.

Upon Finish, the main form now shows that the results can be displayed.

The results screen can then be selected.

Interpreting the results

The first table shows the ranking and weights assigned to the criteria. This is the same table displayed earlier. From this we see which criteria were more important and those not so important.

The second table is the result set for which we are looking. This is the final, calculated ranking for the options. It looks like the vacation to the beach, mountains or lake is the winner, but visiting the relatives is a close second. This means that either of these two are likely to be a good choice.

The last table shows each option by each criterion. This information is helpful to see where bumps and dips in the numbers occur.

Printing the results

To print the results, right click in the report window. Since this is a browser object, the options should be similar to those on your normal browser. Print… is usually one of the options.

The program is complete, working, tested, and ready for public consumption. There are a number of great places to put open source projects for public use. I chose Google code since I am familiar with some of Google’s other tools and software.

Right now I am fighting with the software that creates the setup installation program. And I expect to win this fight shortly, in a  day or two.

The following introductory material explains what is meant by a multiple-criteria, multiple-option decision. Other places in this blog contain similar information. The following is also on my wiki page for this program.

Introduction

Simple decisions are hard enough to make. And sometimes more options don’t make us feel better, rather worse. The decision process becomes exponentially complex. And to add more criteria by which to measure the options — that just makes our heads hurt.

That is where the multiple-criteria, multiple-option decision analysis (MCMODA) tool can greatly help.

This site has the program you can run on a Windows computer. (Apologies to Mac owners. I am not so fortunate as to own a Mac. Wish I did.)

First, a simple decision method for comparison.

First, lets look at simpler decisions and go from there. Many of our choices are simply picking one option among many options. There are several methods for doing this; relying on gut reaction is the most common practice.

A more rational method would probably lead to better results. So we use a binary elimination.

1. To do this, we pile the options up like cards.
2. Then we pick any two options and examine them.
3. We toss one option aside and keep the other option for later.
4. When we finish the first pass, we pick up the keep pile and proceed through it in the same manner as we did the first.
5. This leaves us with a much smaller keep pile.
6. And we continue this over and over, until there is only one option left.

This method uses only one criterion by which to measure all the options. Often our criterion is simply a fuzzy feeling of this I like and that I don’t like.

If we only have options to sift through and select, the process is fairly straight forward.

Now to complicate our decision

What happens if we still have a good number of options, but we then add several criteria? And possibly, some criteria are more important than others.

Let’s take a more complex decision. We have two weeks of forced vacation upcoming. What do we do? First, the options:

• Take a cruise
• Take a closer vacation to the beach or mountains or lake
• Visit some relatives
• Work on finishing that boat in our garage so we can go fishing
• Work for a friend’s business and make some money
• Finish that big home maintenance job and make the family happy

All of those options are worthy goals. But the choice is not so simple as we would like because we have many, competing criteria. So here are the criteria. Each must be stated in a positive way as a group. That is, since we are counting points, the most points wins for each item. But more on this restriction later.

• Which will cost me the least (or put the most money in my pocket)?
• Which is something that more people in the family would like rather than only one or two individuals?
• Which would take the least time to get to and from, to avoid wasted travel time that takes away from our fun time?
• Which is the easiest to do, at hand?

This is called a multiple-criteria, multiple-option decision. The above can be worked out on paper in 15 minutes or so. Wouldn’t it be nice if I could just type in those things, and let the computer assist me to analyze the choice faster and also show the close second and third place options?

That is the reason for the program.

A little history (snooze time)

Over 25 years ago, this method was presented in a conference as a standard decision making method. It was all paper-based. But for decision that had expensive consequences, the time was more than worth the effort. But the paper-based method became more cumbersome as more options and criteria were added.

Then also many years ago, some very important decisions came up that required some careful thought, especially those related to jobs, moving the family, and other life decisions. Paper and pencil weren’t adequate.

I created the first program from a primitive form of Basic. Later, Microsoft Access became available. With some VBA programming and a little work, a simple MCMO program was created. And this program was used through many versions of Access. The program was useful and helped me in many situations.

On this website, http://thethirdway.org, some blogs discussed a variety of decision methods and even provided some spreadsheets to help. It was the MCMO spreadsheet that generated the most interest.

I am not happy with that spreadsheet. Yes, it worked, but definitely not user friendly. It was only a stepping stone to something better and easier to use. So I wrote the program.

Following up

The program should be ready for your consumption within a day or two of this blog. After getting the program out to you, I will provide some sample runs, illustrations, and instructions on how to use it. Any comments regarding the program itself can be made here or at the Google code site.

Have a great day!

The first step in decision analysis: obtain the criteria and the options. The program presents an opening form to do this. (Program will be ready for download by New Year’s day.)

Multiple Criteria, Multiple Options Decision Analysis, Main Form

What’s the difference between Criteria and Options?

One way to decide if something is a criterion or an option is this:

Criteria: attributes of things and actions, a comparative quality, a measure
Options: things or actions

Examples of Criteria used to compare two Options:

• Which costs less?
• Which takes more time?
• Which is easiest to do?
• Which is more pleasing to the eyes?
• Which is more colorful or tasteful?

Examples of Options (things or actions or resulting states of being):

• Buy my favorite new car
• Buy a used car
• Go back to school full-time
• Get a degree from online courses
• A new job
• This couch or that couch, or chair, or picture

After using this program for a while, you get a feeling for the difference between criteria and options. To my knowledge, there is no precisely scientific or mathematical separation of these. But it makes common sense that when comparing a thing or action or state with another, the basis of the comparison is some quality or measure that allows me to judge one from the other.

Criteria are all positive or negative

All criteria should lean the same way. That is, a higher value indicates desirability or undesirability across all criteria. For example, the two following criteria likely conflict–leaning in two different directions:

• Which item costs more?
• Which item will save me the most time?

This appears to say that both costing more and saving time are desirable. Usually, they are at odds with each other. Likely, the first criterion gets reworded as follows:

• Which item costs less?
• Which item will save me the most time?

Likewise, if you are trying to find the most undesirable options within a set of option (a form of negative analysis), then these criteria get reworded in the opposite direction so that the worst option has the highest value.

• Which item costs the most?
• Which item saves the least amount of time?

How to use the main form: the steps

The steps for using this initial form are as follows:

1. To enter a criterion, type it in the text box above the List of Criteria.
2. To remove a criterion from the list, double-click that specific criterion on the list.
3. To remove all criteria, click the Clear All Items button below the list.
4. Add and remove options in the same manner as done for Criteria.
5. If you wish to save your work to this point, (Step 2 on the form) click on Save Criteria and Options.
6. First rank the criteria (Step 3 or Rank Criteria on the form). This will take you to a different form where you select or rate each pair of criteria. This will be described in the blog on that form.
7. Sometimes, there are some criteria that have very low weights. Ranking options against these may be a waste of time. In such cases, return to the first three steps and cull out low value criteria from the rest. You may also want to re-rank the criteria again.
8. Once the criteria are ranked and assigned their weights, you can then rank the options against each criterion (Step 4 or Rank Options for Each Criterion on the form).

The print function is available through the File menu item. There are several reports that are helpful.

• Ranked criteria report: This report lists the criteria by weight in descending order. This report lets you know if there are some criteria that are just not important enough to include when ranking the options later.
• Ranked options report: This report lists the options by summary weights in descending order. This is the summary of all your work and the most important of the reports.
• Ranked options report with details: This report contains the summary report, but contains the weights for each criteria. You can see if there are bulges and dips in the numbers, and you can see what separates the winners from the losers.

A Caveat

This program is for information and entertainment. Every effort has been made to ensure the accuracy and completeness of the comparisons, weights, and process. However, this is only for information. You are responsible for any decisions you make should you base some of your rational on the reports produced here.

And I can’t refund your purchase price. This program will be free.

The spreadsheet was a temporary measure for my personal use. Seeing the response and need for this… well, it is time for the full-blown program. A program is in progress to make this analysis a whole lot easier. The first version will have the following functions:

• It allows the user to add/change/remove both criteria and options.
• It allows the user to select the 3-level comparison (more, the same, or less) or to use a 5-level comparison (much more, more, the same, less, much less)
• Simple output text file or possibly HTML containing all the criteria and their weights, all the options and their weights by criterion, and total weight

Then future releases of this program will include additional features:

• Simple voting on options (one criterion, just a selection from multiple options)
• Simple weighting (one criterion, multiple options, 3- or 5-level comparisons)
• Fancier output file if the HTML is not sufficient, maybe an Excel spreadsheet in its XML format for easy import

The hope is to create a small, executable version that people can use by just copying it to their PC and running it. I am also hoping to set this up as a project on Sourceforge.net or other open source repository so that people can download the program or even get the  source code.

And it will be free. I love freeware. And I feel compelled to reciprocate, return the favor.

I have had this program running for many years as a Microsoft Access database (old version), and it creates some nice reports. This program doesn’t work will with the newer versions of Microsoft Access. And the user interface is not fault-tolerant meaning the  wrong keystrokes result in disaster. In its original state, this program is not for public consumption.

The plan is to have this up and running before New Year’s Day. That way, we can go through our New Year’s resolutions and throw out the unlikely misfits among the bunch.

And if you have some features you want to see, please let me know via comments. These likely won’t make it into the first release, but hopefully queue up requests for the subsequent updates.

In case you don’t get to stop by in the mean time,
Have have a Merry Christmas and Happy New Year.

To explain, I’m not in the market for a one-year supply of food. However, being interested in preparedness and surviving natural or man-made disasters often led to searches for “one-year food supply.”

This latest recent search identified the of the most popular warehouse clubs. And both are really carrying a one-year supply of food.

The good and the bad of mainstream: anyone want a rat

On one hand, we can take comfort that a disaster preparedness food supply is locally available and at a reasonable price. On the other hand, seeing such things in the store is somewhat unsettling.

Consider the rat. In normal times when there’s plenty of food and life is going well, the rat is a pest. Except for pet stores, no other stores sell rats. But let a famine or disaster wipe out the food supplies. In times of famine the butcher shop may carry rats. And rats become a delicacy at premium prices.

In times of plenty, people shop for food weekly and some even daily. When mainstream stores carry long-term food supplies, there must be a market for it. People are thinking about survival in some form or other. People are worried that the food won’t be on the store shelves for their frequent trek to the grocery.

Worrying about the infrastructure

Several years ago, the Southeast experienced a bad drought. The dryness and lack of water was disconcerting. Upon searching for information on droughts, one article showed that this area typically has a major drought every 10 years or so. Our droughts are a common and expected occurrence and we needed to take this into consideration.

The Atlanta metropolitan area has grown so much over the past many years. The water usage and infrastructure to support water supply was fine for most of the time. However, it wasn’t prepared for handling the drought in conjunction with the large growth of the city. What a collision. Some estimated the city had only six weeks of water left when the rains returned. Six more weeks of drought and we would have been out of water.

Imagine a major American city running out of water. How does a city of this size cope without water? Not very well.

A sign of the time

Now that the mainstream stores are carrying the one-year supply of food, I am both glad and worried. Guess I will save up my money and get me one of those, just in case. And hope I never have to rely on it.

The spreadsheet is provided for informational purposes only. Let the buyer beware. I am responsible for nothing, although I take credit for creating this.

Multiple-criteria weighted-ranking spreadsheet (Excel 2007/2010 format)

The way to use this spreadsheet is covered in earlier blogs. However, a few remarks may help.

• This is created for up to 10 criteria and up to 10 options.
• Data is entered in the green blocks. The yellow blocks identify the automatically calculated items.
• If you have less than 10 options, just don’t entere any title or name in the remaining option headers. Likewise, the remaining criteria can be left as-is. I think that the statistics can then be ignored for those options not used. And the unused criteria may average itself out. — But I haven’t experimented with this method.
• For better statistics, enter in all the options and criteria. Then delete all the data or forumulas (not the rows or cells) for the unused criteria. Do the same with unused options, delete formulas but do not delete rows or cells.

This should give you a start. The previous blogs provided details on the steps and how to use this for analysis.

Enjoy!

Step 5. Review, revise, and add hypothesis

After reviewing all the evidence against the hypothesis, you may have a gut feeling that one or more needs to be reworded differently, given a different slant, or even made more specific. Reword and change hypothesis as needed.

Also, new hypothesis or ideas that you want to prove popped into your head. Write those down and put a brief title for them in the column heads.

Step 6. Review, revise, and add evidence.

Some of the evidence will need to be reworded or revised, based on the first pass. Make these changes.

Add more evidence or those “wish I had this evidence” items to the matrix. Some of these will occur during the first scan of the matrix.

Step 7. Fill in the blanks or make changes related to the changed and new items from the previous two steps.

The previous steps modified and added items. It is probably easier to begin with the changed and added evidence. Afterwards, handle the remaining evidence vs. hypothesis intersections that the first part didn’t handle.

Step 7a. Review the current consistent-inconsistent values of each of the changed evidence against each hypothesis. For new evidence, enter the consistent-inconsistent value related to the evidence for each hypothesis.

Step 7b. For each changed hypothesis, review each consistent-inconsistent value that was not reviewed in step 7a. For new hypothesis, enter in the consistent-inconsistent value at each intersection with a line of evidence.

Step 8. Lightly mark out or X out any line of evidence that is consistent with all hypothesis. This evidence is not important to further culling and analyzing of the hypothesis. However, some of this evidence may be required if you are reporting on this analysis. (Hence, don’t obliterate it, making it unreadable.)

Step 9. For each hypothesis, determine if there are sufficient “inconsistent” grades to remove it from further analysis. Again, mark it out but don’t obliterate it.

Step 10. Rank the remaining hypothesis based on the number of the inconsistent marks, and the the weight or importance of evidence to which it is inconsistent. That is, some evidence has much greater importance in the overall analysis and this must be accounted for in your analysis. This ranking should be from the more likely to the least likely.

Step 11. Sometimes, further analysis of the most likely hypothesis is important. This analysis would include time-lines, causality trees, consequences, and so on. When the above steps point to a most likely hypothesis, this means that the hypothesis was not disproved. However, the most likely hypothesis was not proved either.

Further analysis can provide supporting evidence for the hypothesis using different tools. This builds confidence in that hypothesis if the analysis aligns with the hypothesis. This is like running experiments against the hypothesis to see if it holds true under different situations and exposure.

Step 12. What does your intuition or gut feeling say? The purpose of analysis is not to simply make a decision for us. It is to inform our mind and emotions, providing them with a structure and information for decision making.

As with other techniques, we may get a gut feeling part way through these steps; our mind is becoming informed and beginning the decision process. But after it is all over, it is our intuition that we must follow.

If we feel good about the analysis having found the best hypothesis, we are done. If we have the gnawing sense that this is not right, we can revisit the evidence to determine which should be revised, removed, or something added. We may realize that we are missing one more piece of vital evidence.

Likewise, we may want to reword the hypothesis. Or scrap the hypothesis, or add a new one to the list and do the hypothesis testing over.

Next Time …

Hoping to have an example worked up regarding the hypothesis testing. From work scenarios, I hope to find one regarding troubleshooting that would apply.

Since this may take longer than I hoped, I may begin a different topic and return to the hypothesis testing example when the example has been written up and completed.

The following is described in greater detail in Morgan Jones’ book, The Thinker’s Toolkit. It is a great book and easy to read. (I never loan out my copy, never.)

Step 1. Generate hypotheses

We have a natural inclination to jump to conclusions, so this step allows us to give in to this natural inclination. Write up hypotheses as they come to our mind (brainstorming). Hypotheses are statements that explain or give a reason. They may be statements we want to prove or disprove.

From this larger collection, some hypotheses can be combined. Of some hypotheses we may be unsure whether or not to include them. For now they can be written and set aside for the moment. Some hypotheses can be eliminated right away.

Step 2. Create the matrix

A sample, blank matrix is shown in the figure below. Each hypotheses is a column in this table. The hypotheses column heads are labeled A, B, C and so on. Write a shortened version or title for each hypothesis in each column head.

Hypothesis Testing Blank Form

Each row contains a significant item of evidence. These are numbered one, two, three and so on. At this point, we have not listed the evidence, but created places to hold our evidence.

Step 3. List the evidence.

Which of the mass of evidence is significant? Evidence that is unimportant can be ignored. Determining importance is more a matter of context for these hypotheses.

For example, the time of day or the color of the car have no meaning if we’re trying to analyze a business issue. However, if we are solving a crime, time of day and color of the car may be the most important pieces of evidence. So importance of evidence depends on what were trying to prove and disprove, and how well that evidence supports or refutes one or more hypotheses. Often your intuition or gut feeling will guide you as to importance. If undecided, write those items on a separate list that can be used later.

Another important aspect is to overcome the human tendency to satisfice. That is, we tend to stop collecting evidence when we feel an emotional closure during this process. Keep on going. This overcomes our human tendency to see only what we want to see and ignore the rest.

Any evidence of uncertain value on this first pass is put onto a separate list for safe keeping. After completing a round of reviewing all important evidence against each hypothesis, we may want to add some of these uncertain items to our matrix for the next round.

An important category of “evidence” to consider is not actually evidence. What do we wish we knew but don’t yet know? What further evidence would be helpful if we could obtain it? The answers point to further research and analysis that can shed even more light on our hypotheses. What information am I missing that could specifically disprove one or more hypotheses? Where can I find this information? How can I find it?

Step 4. Work horizontally, examining evidence against each hypothesis

Begin with the first row, an item of evidence. Compare this evidence against the first hypotheses and mark the matrix cell as to whether the evidence is consistent, inconsistent, or we just don’t know. Mark the adjacent cell based on how the evidence compares with that hypothesis. Continue until the row is complete.

Then do the same to the next row, then the next row, and so on.

The author says it is best to work in a horizontal manner. For some reason, our minds perform better analysis when holding an item of evidence in our mind and comparing that with each hypothesis. For some reason, we do a less quality job when we hold one hypothesis in mind and compare that with every item of evidence (vertical processing).

Methodical plodding across and down the matrix prevents us from jumping to conclusions and from not carefully considering all the evidence. We avoid our instinctive jumping ahead and making false judgments.

When we say that evidence is consistent with the hypothesis it can mean a few things. It may be that this item of evidence supports the hypothesis. That is, if this evidence is true the hypothesis could be true. At least the evidence does not actively disprove the hypothesis. And maybe if the hypothesis is true, it doesn’t matter if the evidence is true or false. In logic, this is similar to the “don’t care” state. A don’t care value does not invalidate the hypothesis.

We may find the evidence is inconsistent with the hypothesis. This means, if the evidence is true then a hypothesis is likely to be false. This evidence disproves a hypothesis.

The author recommends writing in the matrix. These are helpful as your analysis continues.

For next time

We will cover the final four steps in the next blog. A subsequent blog will contain an example that steps through this entire process.

What about us?

We have ideas, explanations, and reasons of why things happen. We may want to prove a point. These ideas, reasons, points, and so on, we will call hypotheses. Sometimes our goal is to disprove an idea (usually someone else’s).

Too often people begin throwing around hypotheses with vague and weak reasons why they are true. And others come back with different hypotheses and vague, weak reason why theirs are true and the arguments of other people are false. And round and round it goes.

Our penchant is for jumping to conclusions on the first thing that pops into our heads. And so armed with an idea and maybe one or two positive reasons why our idea is true, we vehemently defend our position.

This hypothesis testing may not definitively resolve the argument. At least it can rid the debate of half-baked, crack pot ideas. Hypothesis testing will cull out things that do not stand up against the evidence. It will show which ideas have the most merit and value based on evidence.

The Book

Morgan Jones covers hypothesis testing in detail in his book The Thinker’s Toolkit. (Buy the book; it is very good. And I don’t make a penny from it.) The author’s discussion has excellent examples to step a reader through the hypothesis testing method. If hypothesis testing seems to be of value for your decision-making, do read the book. The author does a much better job than I.

What is a hypothesis? And what do I do with it?

First, the hypothesis is a statement we put forth as true. It could simply be the reason why something happened. It could be our best guess at the situation based on the meager evidence we have in our possession.

The author credits Karl Popper with this: we cannot prove that a hypothesis is true. However, we can show that a hypothesis is false if we find evidence to the contrary.

Hypothesis testing blends deductive logic with inductive logic. It is both divergent and convergent. Many of the best decision-making tools use both divergent and convergent steps. Divergence generates ideas (hypotheses in this case). Convergence then culls out ideas, leaving the few best ones.

Socrates–hypothesis testing made the wisest man in Greece

Stretching it a little, hypothesis testing is something that Socrates practiced. Socrates love going around and urging his victims to spout out some ridiculous hypothesis. Then Socrates would proceed to question and finally prove that their idea was faulty (at best) or possibly stupid (at worst). He won many arguments and lost many friends. But he gave us the Socratic method of coming up with ideas then finding if evidence would refute them.

Hypothesis testing – the 60 second version

These steps are a high-level view and provide a good introduction to this method. Details will be put into the next post.

1. We have a bunch of evidence and come up with one or more explanations or reasons that seem to fit.
2. All this evidence and our ideas are put onto a matrix for this method. That gives the data a structure from which we can carefully examine each and every item. The matrix uses the hypothesis as one axis and the evidence as another. (Next blog posting will show the table and go into details on filling and using it.)
3. For one specific evidence, compare it to every hypothesis. Determine if the evidence is inconsistent with this hypothesis or not. The important point is to disprove, not to prove.
4. Repeat the previous step for each item of evidence.
5. At this point, some hypothesis will be sufficiently disproved and can be removed before the next round.
6. Some new evidence may be available and this can be added to the matrix. Some new hypotheses may also some to mind and those can be added.
7. In some cases, we ask ourselves, exactly what data would disprove one or more of the remaining hypothesis? How can we obtain this evidence? Where can I find it?
8. Then we repeat at step 3.
9. If we are fortunate to obtain sufficient contrary evidence, we may be left with one hypothesis. This will give us a very likely hypothesis. And sometimes we have more than one and just can’t eliminate any more hypotheses. That is OK.

The purpose of the method is to prevent ourselves from jumping to conclusions, ignoring evidence, and having incomplete analysis. We may not come up with the one and only best answer to the question. But we can weed out a lot of ideas and items that don’t fit. Structure helps us bring out our best thinking and avoid the mistakes that can make us look foolish.

Next installment we will cover the detailed step-by-step procedure.