"Wouldn't it be wonderful if the response to the next "anecdote" being passed around the campfire was, 'HOW DO YOU KNOW THAT'S TRUE?' "
paraphrase of a statement by Eric Borneman.

At a recent conference, Eric Borneman and Dr. Rob Toonen both spoke on topics that had as a major theme, the need for hobbyists to reason clearly and critically. Clear reasoning and critical thinking are exceptionally important in our hobby. Probably the most vital component of animal husbandry is the ability to "think" a problem through, and then to rank and subsequently act upon the options that result from that process. I have been writing my essays in this column with the goal of providing assistance for such attentive and reasoning reef aquarists.

The main thrust of my essays in this column has been a discussion of the properties of the various animal groups found in marine reef aquaria. Since its inception, I have provided lists of the major characteristics of each type of animal I have discussed and have expected that, with a bit of reason and thought, most aquarists should be able to use that information either to manage the care of desirable organisms, or to eliminate undesirable ones. However, such processes can only be done when aquarists are both thinking clearly and have at their disposal valid and reasonable information. Unfortunately, valid and reasonable data are exceptionally hard to come by in this hobby, and thinking clearly doesn't seem to found in overabundance, either. What information is available is often hidden by obfuscation or comes from, at best, questionable sources. These problems severely impact the care of all of our animals.

I thought it would be useful to discuss some of the ways in which we can try to cut through the fog of misinformation and myth that surround this hobby. To some extent, I will be following up on themes developed by Eric and Rob, but this topic has been near and dear to my heart for some time, as well. Basically, what I am attempting here is to provide a series of suggestions, or "rules," that, if taken together, should provide some assistance in cutting through this fog of aquarium myth, mysticism, and just plain old "bull."

One thing that is apparent to any reader of the threads posted in online bulletin boards is the all-pervading concept that every idea has equal value. In a real sense this is probably due to the ultimately democratic nature of such forums. Even with some control by moderators, basically anybody can post anything about a given topic. As we know nothing about any person posting, all of their ideas are accepted as having equal merit. Unfortunately, in the big, beautiful, real world, ideas do not all have equal value. Some are good. Most aren't. The same ratio holds true in bulletin board threads. Many of the posts are filled with either factual inaccuracy, or repetition, or inanity or a truly delightful combination of all three.

One of the most difficult things a person has to learn to do is to be able to discriminate between good and bad ideas. Throughout our lives most of the choices we make are not life-or-death decisions. In our hobby, however, making the wrong choice can result in the death of a lot of beautiful animals, both directly and indirectly. So… How can we do better at the choices we make? Well, I will discuss some rules to work with.

Reef keepers, especially those new to the hobby, often assume that a lot is known about the organisms that they are trying to keep. This is not often the case, and as aquarists gain experience, they may gradually come to realize this. Initially, a budding aquarist generally will either take the word of the "instant expert" at their local fish store, or they will purchase one or more "reference" books to rely on. These books soon become infused with the divine authority of a reef god, and are literally treated as gospel. Unfortunately, most of the reef aquarium books are dubious sources of information at best. Anybody can publish one. Packaged in a pretty cover, it will sell. It will sell regardless of the content. These books are generally mixed compilations of lore, myth, and fact. Unfortunately, in most cases it is very difficult to separate the guess, from the fact, from the outright mistake.

One way to assign value to what you are reading is to look for real data. Data are numbers or a series of documented observations with some sort of statistical significance. The reader needs to look for substantiation at every step of the process. For example, when some aquarium author states that such and such a species of coral grows in a given habitat, you should ask yourself, first, "Who identified the coral?" Second, you need to ask yourself, "Who documented it in the habitat?" Third, you should ask, "how many times was it identified and documented?" Once or twice is just not enough to assign such "facts"as habitat preferences. Corals are exceptionally easy to identify. Anybody can put a name on one. Every aquarist can do this. Unfortunately, virtually NO aquarist can do it reliably or accurately. The real problem is not in identifying corals, but rather it is in identifying them correctly. To do this well one needs specialized training, specialized observational skills, observational equipment such as microscopes, a library of references and comparative specimens. Basically, one needs the resources of a major research institution to be able to identify many small polyped corals to any level below genus (See these columns by Eric Borneman; Need Help! Coral ID? Part I. Taxonomy of Stony Corals, Taxonomy in the Reef Aquarium: A Simplified Guide to Basic Level Identification, Taxonomy in Aquarium Corals: Part 3 - Everything Else - Soft Corals, Zoanthids, and Corallimorpharians). A casual hobbyist simply can't do it. Most aquarium authors can't either. They have neither the background, nor the necessary references, nor the necessary equipment. And if somebody needs a lab and lots of references to do this, they are not going to be able to do it "on the fly" as were, during a SCUBA dive or from a photograph.

If a reference book shows a picture of a given animal, and identifies it to species, do you automatically believe the author? If so, why? I don't, unless I see some sort of notation telling me who verified the identification and what scientific or consulting institution they work for. Even those scientists who identify animals as part of their professional research need to have their work verified. I know because, 1) I am a professional taxonomist, and 2) I am often hired to verify the work of other taxonomists. At other times, on other projects, a secondary taxonomist will verify my work. So, when I see an aquarium photo guide listing a lot of species, but listing no verifying authority, I take anything I see in that book with a grain of salt; a small grain of salt about 10 feet on a side.

Likewise, when you read about the requirements of any animal, you should look for a citation to a reference in a scientific journal (and not an aquarium reference) to see where the information comes from. Then, if keeping the animal is critical to you, you should go to a library and examine that reference to verify that the author got the information correctly. You will probably be pleasantly surprised when you find that the author actually did get it right.

The most important thing to remember is to develop a healthy dose of skepticism! Do not accept information without documentation. And make sure the documentation is from a reliable source. You DO need to verify sources, and you should not automatically trust unsubstantiated references.

This skepticism should REALLY be directed at advertisements. In our hobby, many vendors depend on hobbyist ignorance to sell their products. Ignorance, fortunately, is a condition cured by knowledge. For example, when you see a statement like "replace what gets used up," you should ask yourself the following question, "If substance "X" gets used up, what happens to it and where does it go?" The most fundamental natural law is the law of conservation of matter and energy. Matter cannot be created or destroyed, nor can one element be converted into another. So, if you see a statement that some element is "used up" in aquaria, just what does that statement mean? Ask yourself, "How has it been used by organisms?" "Has it been exported from the aquarium?" If it hasn't been exported from the aquarium, it is still there, and it hasn't been "used up;" rather it simply has been moved from one place or form to another. Then you need to ask yourself, where is the documentary evidence that any specific material is used in any manner?

And your favorite reef book ain't the place to go to find documentation!

Unless there have been experiments with actual tests and measurements leading to published data, all you have to go on is hearsay. And if all that hearsay is flouted on advertising copy, then a prudent aquarist should keep their wallet shut. Before you add anything to an aquarium containing living animals, you need to know if what you are adding is harmful or not, and then you need to know if it is necessary. In very small amounts, most materials are harmless in aquaria, but unless they are exported, they are continually cycled and recycled between and within organisms. As more stuff is added to the aquarium, materials can rapidly build up to levels where they can cause problems, either as poisons or as excessive nutrients. So, you should ask for some documentary evidence that the material in question actually is necessary to the well being of any organism.

Some notes about the above, and other, references:
1) Most of these journals have websites that provide abstracts or summaries of current and a variable number of archived issues. These are generally searchable. That way, you can even check from home to see if there are issues with articles you are interested in reading without even making the trip to the library.
2) Not all scientific articles or ideas have equal merit either. However, all true and reputable scientific journals publish articles that have been reviewed and examined by other scientists familiar with either the animals, the techniques used, or the system examined. This "peer review" does not guarantee correctness or "good science," but it significantly cuts down on the amount of grossly inaccurate material for the reader, and while aspects of any article may still be questioned, the degree of "incorrectness" is not likely to kill animals in aquaria.
3) Specifically regarding the journal, Coral Reefs; this is probably the best single source of information for reef enthusiasts. All reef aquarium enthusiasts should become members of the International Society For Reef Studies. It is an exceptional bargain, and supports coral reef science, conservation, and research, and society membership includes subscriptions to both Reef Encounters and Coral Reefs.

In other words, check the data. If you are going to add something to your glass box containing several hundred to several thousand dollars of livestock, don't you think you ought to know what is in it? Take a look at the side of your favorite bottle of "Reef Glop." If it doesn't tell you what is in it, how do you know what is in it and how much? If you don't know what is in it, how can you make a reasoned choice as to whether or not, you should add it? It gets even dicier; several additives have been chemically analyzed and have been shown not to contain the materials that the manufacturer claimed were on the label. (See Shimek, 2001; and online discussions).

So, how do you decide what to add to your aquarium to promote the health of the organisms? Well, there are really three choices. You can believe advertising hype, or you can believe undocumented aquarium references, or you can believe information from research documented in the scientific literature.

Unfortunately, it is a lot easier to believe the advertisements or the undocumented sources than to go to a library and look up information in the scientific literature. Upon such laziness and the subsequent residual ignorance is virtually the entire business of aquarium additives and devices built.

"When my cousin Vinnie added some marinara sauce to his aquariums the corals did better…."

And just how was Vinnie able to tell? This is an example of anecdote. Some folks would say "anecdotal evidence;" however, there is no evidence with anecdote. Anecdote is unsubstantiated or unverified observation generally made by an unqualified observer who often really doesn't know what they are looking at. One of the striking characteristics of the reef aquarium hobby is the reliance on unsubstantiated lore or myth, and its treatment as "God's own truth." In fact, one of the oddities of this hobby is the preference that most hobbyists seem to have for anecdote over documented or scientific evidence. I think this preference for unsubstantiated myth over documentary evidence is due to four factors. First, many of the early references were written with a glib mix of science and anecdote. This gave a lot of anecdotal beliefs an unwarranted aura of believability. Second, while the ease of communication with the internet allows the easy spread of information, most really documentary data are still primarily in print sources, not electronic ones. Consequently, the scientific data that could be used to refute some of the wacky ideas that float around the hobby are not easily accessible to the average internet surfer. Third, most hobbyists simply don't understand the difference between anecdote and experimentally derived or determined data. Finally, many anecdotal procedures or techniques are simple or promote simple clear-cut "cures" to problems. For most folks, there is a profound appeal of this simplicity over the real complexity of our aquaria and the organisms we keep. It is easier to believe in and "understand" a simple cure, even if it is wrong, than it is to believe in the more complicated processes that may actually be occurring.

How is it possible to determine what is anecdote and what is not? Basically, the hobbyist needs to look for some sort of experimental documentation. Once the experimental documentation is found, it should be evaluated. This can be tough to do. Often it is tough to find the actual numerical data, and then it can be difficult to understand. Scientific data, whether it is observational or experimental, is generally expressed, at least in part, with numbers. Unfortunately, a large proportion of the reef aquarist population appears to be uncomfortable with mathematics. Or at the very least, it has been a long time since the individual has had to do any real type of mathematical thinking.

Nonetheless, there are some things to consider which can be used to separate anecdote from actual data. First, one observation does not an experiment make. Look for duplication or replication. Second, examine the data for controls and experimental design. Most experiments are simplified as much as possible. In essence, in doing experimental research, one tries to only have one factor varying at any one time. All other factors are kept as consistent as possible. It is fundamentally impossible to have two biological systems that are identical. Consequently, when organisms are used in biological experiments, the goal is to try to get them as alike as possible, and use statistical methodology to determine if the differences seen in the experiments are real.

Our aquaria are just as difficult to deal with as the natural world, and many of the caveats regarding natural observation and experimentation are just as valid in aquaria as in the real world. For example, one observation of something in a tank is not enough to establish even a correlation between two things. Furthermore, even strong correlations do not say anything about causation. Our systems are multifactorial in the extreme, and it is essentially impossible to do "controlled" experiments in them, if that control is understood to mean the type of controlled experiments possible in a simple laboratory experiment. The typical hobbyist example of, "I added "x" and my tank never looked better," is, plainly and simply useless. It is patently impossible to determine cause and effect from such an observation. Additionally, humans are oriented toward visual stimuli to an extent that may be without parallel in the animal kingdom. This means we tend to judge everything by "how it looks." But, without some independent framework of reference we don't know whether or not to believe what we are seeing, or what it means, in any case. Aquarists would do well to remember the aphorism about books and covers; appearances are deceiving are not verifiable.

There are lies, damn lies, and then, statistics.

The above statement, usually attributed to Mark Twain, seems to sum up most people's attitude toward statistics. In most cases, I think, hobbyists faced with trying to understand some of the common statistics, simply choose to ignore them or to ignore the articles altogether. Unfortunately, this means that they lose out on a lot of the importance of any articles containing statistics, and given that most scientific research is validated with statistics, they miss out on the importance of most research. However, it really is not important to understand the basis for choosing a particular test or how the test is performed. Generally, if the test is inappropriate, it will not get published as the scientific journal editors will require it to be correct before publication. This means all a reef hobbyist has to do is examine the results and note 1) what is being tested and 2) the level of probability.

The levels of probability are often misunderstood. The most frequently published probability is termed "alpha" probability. This is the probability of choosing the wrong answer given the data at hand. Alpha probability is generally expressed as a small number, usually 0.05, 0.01, or 0.001, and referred to as the "level of significance." Usually, for results to be considered to be "statistically significant," they must have an alpha level of 0.05 or less. This means that the probability of finding the results as given is less than 0.05 or one chance in twenty. Phrased another way, if you perform a test that shows a difference occurring with an alpha level of 0.05, you are 95% sure that you would not get such results by random chance alone. The lower the alpha levels, the greater the level of certainty, so if the alpha level is 0.01, the level of certainty is 99%, and so on. If you see numerical values, such as averages, compared without a statistical analysis, do not put much reliance on the those data unless the author makes a reasonable argument as to why there is no statistical treatment.

You assume, WHAT?

Everybody makes assumptions, and those assumptions then become part of their arguments and discussions. Unfortunately, these assumptions are seldom clearly and explicitly stated. It is important for readers of such discussions to be aware of these assumptions, otherwise any statements that are made will mean different things to different people. Some of the assumptions that are made by aquarists are probably harmless, such as the belief that vitamins defined by their use in people will have beneficial effects in coral reef animals. As ridiculous as this is, addition of low levels of human vitamins probably have no long-term deleterious effects in the treatment of our animals. Other assumptions are not so benign. Many aquarists seem to believe that being in an aquarium fundamentally changes the way these animals metabolize and physiologically respond to problems. Such an assumption leads to statements such as, "I don't need information about what these animals do in the real world; I need to know how to take care of them in my tank." The understood or, better-stated, misunderstood, assumption here is that the animal can somehow tell it is now in an aquarium and that it changes its physiology accordingly. Animals cannot do this. Wrongful assumptions such as this simply kill animals, and are the basis of many of the inappropriate pieces of advice that seem to be prevalent in the hobby. We all have assumptions in our thought processes, but it is important to see that such assumptions are grounded in reality.

Aquarists need to build up their skepticism of unsubstantiated claims, unfounded anecdote and advertising hype. As a group, we spend far too much time, money and effort on unproductive and counterproductive efforts in aquarium care. This misdirected and misguided effort directly effects the well being of aquarium organisms and contributes to the difficulties many aquarists have in keeping what are, fundamentally, hardy organisms. If we can develop a series of rules and some mental discipline, we should be able to ignore the ridiculous, and concentrate our efforts toward the better care of our boxes of water and their included critters.

My thanks to Eric Borneman and Rob Toonen for the inspiration for this essay, and to Eric Borneman and Skip Attix for their reviews which improved it.

References Cited:

Shimek, R. L. 2001. Necessary Nutrition, Foods and Supplements, A Preliminary Investigation. Aquarium Fish Magazine. 13: 42-53.