Envirologix: Putting Science to the Test

USDA GIPSA Guide: Sampling for the Detection of Biotech Grains - Part 3

The Impact of Sample Size on Risk Management

In the section on sampling theory, the range of likely sample estimates was shown to decrease as the sample size increased. Several assumptions underlie the sample size effects discussed so far. One assumption is that every kernel in a sample can be determined as biotech or non-biotech without error. The variability shown in the sample estimates assumes only sampling variability. No allowance for error from sample preparation or from analytical method has been incorporated into the estimate ranges.

Measurements associated with grain lots are usually given as percent by weight. The estimates given in the previous discussions expressed the estimates as the percent of the total number of kernels. The percent by kernel count and percent by weight would only be the same if the kernels were all the same size, but kernels are not uniform. Percent by kernel count is, however, usually a reasonable approximation to percent by weight. Kernel counts can be converted to approximate weights by using average kernel weights observed from typical market samples.

The type of measurement is also a consideration in determining the sample size. The analytical methods available for detecting biotech grains may be used to make qualitative or quantitative tests on a sample. A qualitative test can be used to screen lots by providing information on the presence or absence of biotech varieties. Quantitative tests may quantify the total amount of biotech grain, the amount of individual varieties in a sample, or the percentage of biotech or "non-native" DNA or protein present relative to non-biotech grain.

When sampling is used in the measurement of some characteristics of a lot, the content of the sample will likely deviate from the lot content. The buyer accepts some risk because the sample may overestimate the quality of the lot. The buyer may assume that the quality of the lot is better than it actually is. Likewise, the seller accepts some risk that the sample may underestimate the quality of the lot. In this case, the seller is delivering better quality than the sample reflects.

Ideally, buyers and sellers would agree to use a sampling plan that provides acceptable risk management. A contract may specify a certain quality level. However, due to sampling variation, a seller may have to provide better quality to have grain lots accepted most of the time. Sellers should choose a quality level that they want to have accepted most of the time, say 90% or 95%. This level is sometimes called the acceptable quality level (AQL). Likewise, due to sampling variation, the buyer may sometimes have to accept lower quality than the contract specifies. Buyers should choose a lower quality level (LQL) that they want to accept infrequently. This LQL may be acceptable 5% or 10% of the time. The ideal sample plan would meet both the AQL and the LQL.

A single sample with a qualitative test gives little flexibility for choosing
an AQL.

Quantitative tests, when available, and multiple sample plans provide more
flexibility to choose both an AQL and LQL.

Sample Size – Qualitative

In reality, all analytical methods have limits of detection. For purposes of this section, the assumption is a qualitative test will detect the presence of a single kernel in a sample, regardless of the size of the sample. A positive result does not tell how many biotech kernels are in the sample, only that at least one biotech kernel is present in the sample. To choose a sample size, the acceptable and unacceptable concentrations must be decided upon. Since samples are subject to sampling error, acceptable lots may be rejected, and unacceptable lots may be accepted just by chance. Buyers and sellers must agree upon acceptable risk.

The following chart shows probabilities associated with different sample sizes (based on kernel count). The horizontal axis gives possible concentrations in lots. The vertical axis gives the probability of observing no biotech kernels in a sample randomly selected from a lot. Curves are given for various size samples.

The above chart shows probabilities for sample sizes of 60, 120, 200, and 400 kernels. If the desired concentration on the lot is not to exceed 5.0 %, a sample size of as little as 60 kernels may be satisfactory. Based on a 60 kernel sample, there is a 95 % chance of rejecting a lot at a
5 % concentration. If 1.0 % is the desired maximum concentration in the lot, a sample size of 400 kernels would be more appropriate.

The following chart shows probabilities associated with sample sizes of 800, 1200, 2000, 3000, and 5000 kernels. Larger sample sizes are used only when low concentrations of biotech kernels are acceptable. For example, if 0.1 % concentration is the desired maximum, a sample between 3000 and 5000 kernels would be recommended.

Thus, testing for lower concentrations of biotech grains requires larger
sample sizes.

Sample Size – Quantitative
Quantification of the percent biotech grain in a lot is much more problematic than with qualitative testing. As mentioned in the introduction, sampling variability is only one source of error in measurements. Sample preparation and analytical method can be two significant sources of error in the detection of biotech grains. The currently available technologies employed for detection, the Polymerase Chain Reaction (PCR) and Enzyme-Linked Immunosorbent Assays (ELISA) have inherent difficulties in producing consistent and accurate quantitative results. PCR measures the genetic material, or DNA, associated with the inserted DNA. ELISA tests, on the other hand, measure the protein expressed by the foreign DNA. Both methods present significant challenges in converting the amount of DNA or the amount of expressed protein into the percent of biotech grain by weight. The overall variability of quantitative results, therefore, will be affected by analytical methods as well as sample size. Sample size typically will have little influence on the sample preparation or analytical method. Sample preparation and analytical method are significant sources of error, and increasing the sample size will not reduce the overall variability in measurements as much as expected.

The effect of sample size alone on sampling variability can be examined. The sample is used to estimate the percent concentration in the lot. Using percent by kernel count as an approximation of percent by weight, probability curves can be computed to examine the probability of accepting lots for which a maximum concentration has been specified. The following chart shows the probabilities associated with a 0.1 % maximum allowed level of biotech kernels in the sample.

For this example, a lot with less than 0.1 % concentration of biotech kernels is defined as acceptable. The curves on the plot give the probabilities of accepting different lot concentrations with five different sample sizes. For concentrations above 0.1 % the curves give the probability of accepting an unacceptable lot. This probability may be called "buyers risk" because this is the chance that the buyer will get an unacceptable lot. For lots with less than 0.1 % concentrations, the area above the probability curve represents the chance of rejecting an acceptable lot. This may be called the "sellers risk" because this is the chance that the seller will have an acceptable lot rejected.

The ideal sampling plan will minimize both the buyers and sellers risk. Unfortunately, no one sampling plan will produce both objectives. Increasing the sample size can reduce both buyer risk and seller risk. Theoretically, the only limiting factor on the sample size is the lot size. Sample size is often determined by a compromise between the seller and buyer risks and the cost of taking and processing a sample. The 5000 kernel sampling plan in the chart above allows the buyer to conclude that a 0.2 % lot has less than a 10 % probability of being accepted and the seller to conclude that a 0.05 % lot has less than a 10 % probability of being rejected. If the seller is satisfied with an AQL of 0.05 %, and the buyer is satisfied with a LQL of 0.2 %, a 5000 kernel sample with a maximum of 0.1 % biotech kernels may be an acceptable plan to both parties.

The following chart gives sampling plans that allow a maximum of 1.0 % biotech kernels in the sample.

Again, increasing the sample size will reduce both the buyer and seller risks. The following chart shows sampling plans that allow a maximum of 5.0 % biotech kernels in the sample.

Estimates for higher percent mixtures will be less precise for the same size
sample. However, the buyers and sellers may be willing to accept less precise
estimates for these higher mixtures.

Multiple Sample Plans – Qualitative Testing

Previous sections discussed the effects of sample size with qualitative and quantitative testing. One way to express the effect of sample size with qualitative testing is that, for any given concentration, the probability of a negative result decreases as the sample size increases. The only time this is not true is when the lot concentration of biotech grain is zero. If the buyer of a lot has a zero tolerance for biotech grain, taking the largest sample possible will best serve the needs of the buyer. The buyer can be reasonably certain that a lot with a high concentration of biotech grain will be rejected.

A single large sample serves the buyers interests well. However, some buyers may be willing to accept some low concentrations but unwilling to accept high concentrations. Sellers of lots with low concentrations would like to have high probabilities of testing negative. Decreasing the sample size will increase the chances of a negative result on low concentrations. Unfortunately, decreasing the sample size increases the chance of a negative result with higher concentrations. When a low concentration is acceptable to the buyer, a single qualitative test may not serve the interests of both the buyer and the seller. An alternative is to implement a multiple sample plan.

Multiple sample plans specify that a certain number of independent samples will be selected from the lot and each sample is tested. The buyer will accept the lot if certain combinations of positive and negative test results are obtained. For example, the sample plan may specify that five samples of 100 kernels will be selected from a lot. If no more than three positives are obtained on the five tests, then the lot is acceptable.

The components of a multiple sample plan are the number of samples, the size of each sample, and the maximum number of samples testing positive. Changing any one or more of these parameters affects the probability of acceptance. Buyers and sellers can choose a plan based on the risks they are willing to assume, and on the cost of conducting the tests.

With the example of five samples of 100 kernels, the maximum number of positives specified in the plan could be one, two, three, or four. The following chart gives the probabilities of accepting lots with these four plans. Increasing the maximum number of acceptable positives will result in higher concentrations being accepted.

By manipulating all three parameters, the shape of the probability curves can be changed considerably. The following graph compares a six-sample plan with a 60-sample plan. Both plans have low probabilities of accepting a 1.0 % concentration. The probability of accepting a 0.5 % concentration sample, however, is considerably higher for the 60-sample plan than for the six-sample plan.

(The above multiple sample example is& from "Statistical Considerations in Seed Purity Testing for BioTech Traits" by Kirk M. Remund, Doris A. Dixon, Deanne L. Wright, Larry R. Holden, which is currently in review for publication in Seed Science Research.)

Multiple sampling plans can be used to balance the risks between buyers and sellers when low concentrations are acceptable to buyers.

Sample Preparation: Obtaining A Portion for Analysis
Samples are typically collections of kernels from a lot. The discussions on

sample size have been presented thus far as if each kernel in the sample was measured independently. The major analytical technologies for detecting biotech grains, PCR and ELISA, usually do not process individual kernels but rather make a measurement on a preparation from the sample.

As mentioned in the introduction, sampling variability is only one source of error in measurements. Sample preparation and analytical methods are two other significant sources of error. Reducing a sample to a portion for analysis is often necessary to meet method limitations. For accurate analysis, the sample portion analyzed must be representative of the lot submitted. Preparation of a sample for analysis must include grinding and mixing of the grain prior to subdivision. Grinding will produce more uniform subsamples for analysis. For example, a representative three pound (1360 grams) composite sample would contain 8500 soybeans. If this composite has 1% biotech content, that would mean about 85 biotech soybeans would be present. If a standard divider is used to reduce the unground sample to an analytical portion of 60-70 beans, not all of the resulting 128 possible sample divisions would have a biotech soybean. If a hypothetical distribution of one soybean per subsample were assumed, then only 85 of the 128 possible "cuts" would test positive for the presence of biotech grain. Forty-three subsamples would test negative. Realistically, however, such a distribution is not likely. The following table and figure demonstrate the theoretical distribution of biotech kernels in the 128 sample divisions.

		Distribution of Kernels
	Kernels	Samples	Percent
	0	65	50.9
	1	44	34.7
	2	15	11.5
	3	3	2.5

It is interesting to note that, for this example, more than 50% of the samples will test negative. Quantitative test results, assuming no other sources of error, would range from 0 to 3.4%. None of the samples would give the analyst the "right" answer.

Thorough grinding and mixing would give a much different distribution of the analyte, producing more consistent results from subsample to subsample. The analyses to be performed must be considered and may be the determining factor with respect to particle size. In a plant seed, the DNA and much of the protein is concentrated in the embryo, and the embryo may only be 10% of the weight of a seed. In the 1360 gram (3 lb.) sample above, the embryonic material from genetically modified soybean seeds will be only one-tenth of 1%. For PCR analysis, where analytical sample sizes are routinely only one gram, research studies have shown that a particle size of less than 200m produces a homogenous sample. Therefore, grinding a representative composite sample to an appropriate particle size, followed by thorough mixing, will minimize sample preparation errors. Analytical results on subsamples of this homogenous sample will be much closer to the actual content (1%), as shown in the example below.

Each 1 gram subsample should contain about 250,000 soybean particles. Although one percent of these particles would be from biotech soybeans, only about 250 of these would actually contain biotech DNA. Using a normal approximation, 97 % of the analytical results would be in the range of 0.86 to 1.14 %.

ELISA testing, which routinely uses larger sample sizes, may not need as fine
a particle size as PCR testing. Studies should be conducted to validate sample
preparation protocols to assure suitability for the analyses to be
performed.

Cleanliness in Sample Preparation
Carry over of materials from one sample to another takes on an even greater significance during sample preparation prior to analysis. Due to the sensitivity seen with many methods for detection of biotech grains, care must be taken to avoid transferring materials to subsequent samples. Whole grains, dust and residual matter must be removed from all equipment. Grinders should be cleaned through vacuuming of dust, washing with soap and water or solvents, or a combination of appropriate cleaning methods for the specific grinder in use. Sample dividers and mixers must also be thoroughly cleaned. Analysts should verify the equipment cleaning process is appropriate to prevent cross contamination. Many of the analytical techniques practiced for detection of biotech crops today can detect levels lower than 0.1%. Physical separation of sample preparation operations from analytical operations is also highly recommended to avoid contamination of sample extracts.

Selecting a Sampling Protocol to Minimize Risk

Sample size, theoretically, is selected to best meet the needs of the buyer and seller. Selecting a sample size often involves a compromise between precision and cost of analysis. In measurement systems where kernels are processed individually, the cost of processing a sample increases in proportion to increases in sample size. For these systems, selecting the smallest sample size that provides acceptable precision is the most cost effective sample size.

Many measurement systems process and measure bulk samples. In these systems, the cost of processing a sample may not increase in proportion to increases in the sample size. Processing a large sample may cost only slightly more than processing a small sample.

PDF files, where provided, require Adobe Acrobat Reader.

- up -