Process Variation and Stochastic Design and Test
Statistical Test Compaction Using Binary Decision Trees Sounil Biswas and Ronald D. (Shawn) Blanton Carnegie Mellon University
plane boundaries between classes in the Tkept hyperspace to maximize the distance between the class boundaries and the measurement data. However, this optimization step in SVM-based classifiers requires specifying the shape of these class boundaries (linear, Gaussian, exponential, and so on). This assumption can add error to Fi(Tkept). Classification based on neural networks uses several simple functions of the Tkept measurement values. This classification technique organizes these simple functions as a network of nodes, where each node is called a perceptron. A network of perceptrons results in a complex nonlinear classification function that describes Fi(Tkept). For example, a neural network representation of the binary function (f: 1 ≤ x ≤ 3 → True) can simply be the logical AND function g of two binary functions (f1: x ≥ 1) and (f2: x ≤ 3). The three perceptrons in this example are f1, f2, and g, where f = g(f1, f2). During the process of deriving a neural network representation of Fi(Tkept), the algorithm used to derive the neural network iteratively updates the perceptrons’ parameters to improve the prediction accuracy of Fi(Tkept) until maximum classification accuracy is reached for some sample data typically called training data. Stratigopoulos and Makris discuss an application of neural networks to test, where they examine the correlation between alternate-test measurements and the pass/fail outcomes of fabricated parts.5 Derivation and training of a neural network, however, is an NP-complete process,6 and thus could require significant computation time. In this work, we use binary decision trees (BDTs) for statistical test compaction, because they have the following properties. First, decision trees require no assumption on the type of correlation (if any) that exists between Tred and Tkept. This makes it possible to derive a more accurate representation of Fi(Tkept) from the col-
Editor’s note: Integrated, heterogeneous devices must deal with process variation, and the cost of testing such devices often exceeds that of digital circuits. The authors use statistics to cross-correlate different tests and find the minimum set required to guarantee product quality while reducing test time and cost. —T.M. Mak, Intel
EXPLICITLY TESTING an integrated, heterogeneous device for all its specifications is very costly. A new test methodology that can minimize test cost while maintaining product quality and limiting yield loss is needed (see the “Statistical learning approaches in test” sidebar). We are developing a statistical learning methodology based on decision trees to compact the complete specification-based test set of an integrated device by eliminating redundant tests. A test is deemed redundant if we can reliably predict its output using other tests that are not eliminated. We use binary classifiers to identify specificationbased tests whose pass/fail results can be accurately predicted through measurements from kept tests (other specification tests that are somehow deemed necessary through analysis or by expert opinion). We call the set of tests that can be accurately predicted redundant tests, and we denote them as Tred; we denote the set of kept tests as Tkept. Therefore, given a set of specification tests, T = {t1, t2, …, tl}, where l is the number of tests in T, we can express the pass/fail result yi of each redundant test ti ∈ Tred as function yi = Fi(Tkept), where T = {Tred ∪ Tkept}. We call the process of identifying Tred and Tkept using statistical interpretation of test data statistical test compaction. Binary classification techniques that might be suitable for deriving Fi(Tkept) include, for example, support vector machines (SVMs),1 neural networks,2 and decision trees.3 SVM-based binary classifiers are especially popular, and we have used them for statistical test compaction.4 In such classifiers, the goal is to derive hyper-
452
0740-7475/06/$20.00 © 2006 IEEE
Copublished by the IEEE CS and the IEEE CASS
IEEE Design & Test of Computers
© 2006 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
lected test data. Also, deriving a decision tree model for Fi(Tkept) simply involves partitioning the Tkept hyperspace into hypercubes, which is a polynomial time process of complexity O(n2 × k3), where n is the number of tests in Tkept, and k is the number of parts in the collected data.7 Therefore, the computation time required for creating a decision tree can be considerably less than the time required for training a neural network. However, the application of a statistical test compaction methodology to a commercial device requires very high prediction accuracy, which is not achievable by simply using a typical decision tree model of Fi(Tkept). Our work involving SVM-based classifiers4 and the work of Stratigopoulos and Makris using neural networks5 also fail to meet this requirement. Consequently, to achieve a commercial device’s accuracy demands, we employ several enhancements not inherent to decision trees. These enhancements include scrubbing the collected data to eliminate outliers, reducing drift between individual wafers and wafer lots due to process variations, and applying principal component analysis (PCA) to derive relevant measurement combinations. We also use guard banding of the specification boundaries and hypercube collapsing to further improve prediction accuracy. Some of these enhancements, such as hypercube collapsing, are novel contributions to statistical test compaction. Test compaction results produced for a commercial microelectromechanical systems (MEMS) accelerometer are promising because they indicate that our methodology makes it possible to eliminate an expensive mechanical test for the accelerometer.
Statistical learning approaches in test The standard approach of explicitly testing an integrated, heterogeneous (mixed-signal, analog, microelectromechanical, and so on) device for all its specifications can be prohibitively expensive. There’s now a growing trend to use statistical learning involving test data from fabricated chips to reduce this test cost. These attempts have mainly focused on regression techniques, in which the goal is either to predict the value of one device measurement using others1-3 or fit the parameters of predetermined relationships between the device measurements.4,5 For example, some researchers use measurements from alternate tests and regression to predict device specifications1 or parameters.2 Alternate tests are simplified stimuli and response measurements that are significantly less expensive than their specification-based counterparts. Other researchers perform Monte Carlo simulations of the device design to derive the device measurements’ joint probability distribution (JPD).4,5 They then use this JPD to determine parameter values for the function that relates the measurements. Brockman and Director aim to ensure that the function can accurately predict test measurements.4 Milor and Sangiovanni-Vincentelli want to create a function that can predict out-of-spec measurements when a defect exists.5 Go/no-go testing, however, requires only modeling the pass/fail outcome for the device under test. This is far simpler than predicting a test’s actual measurement value using regression or some other technique. In other words, go/no-go testing is a binary classification problem, so it’s best to use statistical learning based on binary classification to predict the corresponding test’s pass/fail outcome.
References 1. A. Chatterjee and R. Voorakaranam, “Test Generation for Accurate Prediction of Analog Specifications,” Proc. 18th IEEE VLSI Test Symp. (VTS 00), IEEE CS Press, 2000, pp. 137-142. 2. V. Natarajan, S. Bhattacharya, and A. Chatterjee, “Alternate Electrical Test for Extracting Mechanical Parameters of MEMS Accelerometer Sensors,” Proc.
24th IEEE VLSI Test Symp. (VTS 06), IEEE Press, 2006, pp. 665-673. 3. L.-C. Wang et al., “On Path-Based Learning and Its Applications in Delay Test and Diagnosis,” Proc. 41st Design Automation Conf. (DAC 04), ACM Press, 2004, pp. 492-497. 4. J.B. Brockman and S.W. Director, “Predictive Subset Testing: Optimizing IC Parametric Performance Testing for Quality, Cost, and Yield,” IEEE Trans.
Semiconductor Manufacturing, vol. 2, no. 3, Aug. 1989, pp. 104-113. 5. L. Milor and A.L. Sangiovanni-Vincentelli, “Minimizing Production Test Time to Detect Faults in Analog Circuits,” IEEE Trans. Computer-Aided Design of
Integrated Circuits and Systems, vol. 13, no. 6, June 1994, pp. 796-813.
Proposed methodology The n tests in Tkept describe an ndimensional hyperspace. Test response data for each fabricated instance or part is a data point in the Tkept hyperspace; all the test data together form a distribution
November–December 2006
in that hyperspace. Parts that pass each test ti ∈ Tred within the Tkept hyperspace describe passing subspaces, whereas the failing parts represent failing subspaces.
453
Process Variation and Stochastic Design and Test
Our goal in deriving Fi(Tkept) is to separate these two types of subspaces using our BDT-based statistical learning method. The test response data we use to derive this BDT representation of Fi(Tkept) is called the training data. However, the training data might not reflect future data, because of inaccuracies from measurement errors, a lack of adequate training data, defects, and so on. This means Fi(Tkept) could achieve 100% accurate predictions for the training data yet exhibit high misprediction for future data. To remedy this shortcoming, we use another set of collected data, the validation data, to check the model pass/fail prediction accuracy. We measure the prediction accuracy by determining the fraction of passing or failing parts (from both the validation and training data) mispredicted by Fi(Tkept). We then use this accuracy to determine whether the tests in Tred are indeed redundant. In other words, if the misprediction error of Fi(Tkept) is lower than a preset threshold, the choice of Tred is deemed correct. On the other hand, if the misprediction error is higher than the preset threshold, the tests in Tred are not redundant, meaning we must redefine Tred or abort the test compaction process altogether.
Binary decision trees We adapted the terminology used in this section from Rokach and Maimon.3 A BDT is a classifier expressed as a recursive partitioning of the Tkept hyperspace. Each nonterminal vertex in the decision tree represents a partition, which is an (n–1)-dimensional hyperplane that separates the Tkept hyperspace into two subspaces at a particular test measurement value vi,j for a test ti ∈ Tkept. Test response value vi,j is called a decision value, and the corresponding vertex in the tree is called a decision vertex. The subspace of parts resulting from the partition with measured values less than vi,j is the left-child vertex; the subspace of parts with values greater than or equal to vi,j is the right-child vertex. Each test ti ∈ Tkept can have several decision values {vi,1, vi,2, …, vi,m}, meaning a dimension can have multiple partitions. Each terminal vertex in a decision tree is ideally a homogeneous hypercube containing only passing or failing parts. Decision tree construction uses characteristics from a sample of manufactured parts for selecting a test ti ∈ Tkept to partition the hyperspace. The partitioning of the hyperspace with various test measurement values continues until all parts in the sample are partitioned (classified) into homogeneous, leaf-level vertices for all the tests in Tred. Once the tree derivation algorithm has obtained the BDT representation of Fi(Tkept), we deter-
454
mine the prediction of a future chip as pass or fail simply by traversing the tree from its root vertex to a terminal vertex according to the chip’s measurements for the tests in Tkept. Deriving a BDT representation of Fi(Tkept) is a two-step process, involving tree construction and tree pruning. During depth-first tree construction, the tree construction algorithm evaluates the capability of test measurement value vi,j to homogenize the subspace. The algorithm examines the homogeneity of a test and a measurement value vi,j as a candidate partition if the current vertex has at least two parts m and n (one passing and one failing) with measurement values ρi,m and ρi,n, respectively, such that ρi,m ≤ vi,j ≤ ρi,n or ρi,n ≤ vi,j ≤ ρi,m is true. From the test measurement values evaluated, the tree construction algorithm selects, as the vertex’s decision value, the vi,j that maximizes the separation of passing and failing parts in the subspace. The depth-first construction process continues until all training parts are homogeneously partitioned or until further partitioning of the Tkept hyperspace does not improve accuracy. We gauge the derived decision tree model’s prediction accuracy by observing the prediction error of Fi(Tkept) in the validation data. We can further improve the prediction accuracy by “pruning” the tree on the basis of how it classifies the validation data. During a breadth-first traversal that begins with the n – 1 level, the tree-pruning algorithm analyzes each vertex to determine if eliminating the partition and converting the corresponding left- and right-child trees into a single passing or failing hypercube would improve the prediction accuracy. For cases in which the tree-pruning algorithm determines that the prediction accuracy would improve, the algorithm replaces the decision vertex and all its descendants with a single passing or failing terminal vertex, depending on the one that leads to the greatest improvement. This is called the decision node deletion. In addition, the algorithm analyzes each decision vertex to determine whether a small perturbation of the decision value vi,j can improve the prediction accuracy; if so, the algorithm employs this perturbation. We call this decision value shifting. These two tree-pruning operations, decision node deletion and decision value shifting, together can lead to a more accurate Fi(Tkept). Figure 1a shows an example data distribution for a Tkept = {t1, t2} hyperspace. The lines in Figure 1a partially separate passing and failing parts; the dotted line shows the last partition chosen. Figure 1b shows the resulting, partially formed decision tree. The shaded region indicates the tree vertices from the dotted-line partition in
IEEE Design & Test of Computers
Measurement t2
Measurement t2
A
b1
B
C
a1
a3 a2 Measurement t1
Measurement t1 (a)
(a)
t2 = b1
t1 = a1
Left
t1 = a3
t1 = a2
Left
Left
Right
Measurement t2
A
B
Right
C Measurement t1
(b) (b) Figure 1. An example of a data distribution partially separated in the kept test Tkept = {t1, t2},
Figure 2. Possible misclassification of a future
hyperspace (a), and the corresponding partially
failing part (circled triangle) due to inadequate
formed binary decision tree (BDT) (b). Circles
coverage of failing subspaces by the training
represent parts that pass Tred; triangles
data (a), and the collapsing of passing
represent parts that fail Tred. Dotted line in (a)
hypercubes (shaded subspaces) to eliminate
denotes the last partition chosen in the decision
this error (b).
tree derivation process. Shaded region in (b) represents new vertices added to the decision tree by the last partition.
Figure 1a. The subspace corresponding to the new left child contains only passing parts; thus, the child is a passing terminal vertex of the tree. However, the subspace corresponding to the new right child contains both passing and failing parts and hence requires further partitioning. Therefore, the right child is a nonterminal vertex in the tree.
Hypercube collapsing The decision tree representation of Fi(Tkept) is simply the partition of the Tkept hyperspace into hypercubes that separate the passing and failing subspaces. However, training data collected from a high- or low-yielding fabrication process can result in an insufficient number of
November–December 2006
training parts from one class. Consequently, some partitions necessary to achieve an acceptable level of prediction accuracy might be absent. As a result, some portions of the passing or failing subspaces could be erroneously included in hypercubes of the opposite kind, leading to misclassification of a future data point (a manufactured part) falling into that hyperspace. We are particularly interested in defective-part misprediction when the training data is from a high-yielding process and when portions of the failing subspaces are likely incorrectly classified as passing. Figure 2a shows an artificial collection of training data for a Tkept = {t1, t2} hyperspace with high yield. The shaded regions represent the passing hypercubes in the Tkept hyperspace. The circled triangle represents a failing part that was misclassified because the training data did not sufficiently cover the failing subspaces. We have observed that for a sufficiently large sample
455
Process Variation and Stochastic Design and Test
Measurement tb
Predicted pass defect escape
tb specification boundary Pass/fail decision boundary on ta
(a)
Measurement ta
Measurement tb
Guard-band device BDT model II Guard-band perturbations BDT model I
tb specification boundary Pass/fail decision boundaries on ta due to guard banding I
sponding to the partitions along the path. For a passing terminal vertex, the encountered decision values define the bounds of the hypercube associated with the vertex. Any test not included in the path from the root to the terminal vertex manifests itself as a missing bound that reduces the corresponding hypercube’s dimensionality and unnecessarily increases its size. Adding partitions for the missing tests on the basis of which failing parts are present in the hypercube will collapse it. However, the absence of failing parts in the vertex makes identifying decision values for these missing tests impossible. Therefore, instead, the algorithm derives decision values for the missing tests from the passing data in the vertex. For the worst-case scenario, with n tests in Tkept and k passing parts in the collected data, hypercube collapsing creates n × k additional decision values. Derivation of the decision tree model with hypercube collapsing is still polynomial, but with complexity O{(n2 × k3) + (n × k)}. Also, the increased complexity of using the tree with the collapsed hypercubes is at most n – 1.
II
Specification guard banding Measurement ta
(b)
Figure 3. Failing part misprediction due to overlap of passing and failing parts when the test response data is projected to the Tkept = {ta} dimension (a), and elimination of the misprediction using our two-model approach of specification guard banding (b).
of high-yielding training data, passing parts in the training data adequately cover the passing subspaces. Therefore, to guard against the expensive scenario of mispredicting a defective part, we assume that the portions of the passing hypercubes that don’t include any passing part are failing subspaces. So, we “collapse” the passing hypercube boundaries to coincide with the passing-part data in the hypercube, and we denote any remaining subspace as failing. We will then classify all future parts residing in any of these new failing subspaces as defective. These parts cannot contribute to the total number of defective parts being mispredicted, but they can lead to an increase in yield loss, meaning a passing part will be mispredicted as bad. Figure 2b shows the result of collapsing the passing hypercube bounds from Figure 2a. The mispredicted failing part is now correctly classified. Our algorithm performs hypercube collapsing as follows. Each path from a decision tree’s root vertex to a terminal vertex includes several decision values corre-
456
Because of several limiting factors that include, for example, finite training data and errors in test response measurements, Fi(Tkept) will likely misclassify some future fabricated parts. Although completely eliminating the classification error is impractical, we hope to further reduce it by guard-banding the specification boundaries for each test ti ∈ Tred. We formulate guard bands as follows. We reevaluate each training part’s pass/fail attribute after perturbing the specified range of measurement values deemed passing for each test ti ∈ Tred to obtain two new sets of training data. We derive the first set by reducing the acceptable range of measurement values for each test ti ∈ Tred, possibly causing us to treat some passing parts as failing in this data set. We obtain the second data set by expanding the acceptable range for each test ti ∈ Tred, in this case possibly leading us to treat some failing parts as passing in this data set. We then construct two separate decision trees for these two training data sets, and together they constitute Fi(Tkept). We subject each new part to both decision trees for prediction. If the two models classify a part identically, we place higher confidence in the decision and accordingly classify the part as passing or failing. On the other hand, if the models disagree, the prediction uncertainty leads to a guard-band classification—that is, we conclude that we cannot make a high-confidence pass/fail prediction for the part solely on the basis of Tkept measurements. Figure 3a shows the possible misprediction error due to the overlap of the pro-
IEEE Design & Test of Computers
Measurement t2
jections of passing and failing parts onto the Tkept = {ta} hyperspace. Figure 3b demonstrates how guard banding eliminates this misclassification error by placing these overlapping parts into a third, guard-band class. Depending on the number of expected devices in the guard-band subspace and the cost, we can test guard-band devices further to determine their true pass/fail status.8 Alternately, we might decide that the devices in the guard-band subspace are good, bad, or even of lower grade, depending on the application’s quality requirements.
A
(a)
Measurement t1
The collected test measurements for Tkept, though acceptable in terms of the specifications, can include some anomalies. For example, measurements from one wafer to another or from one lot to another can “drift” (that is, measurements from all parts in one wafer or lot might be shifted by the same value with respect to equivalent parts in some other wafer or lot). In addition, there could be outlier parts, which are acceptable according to the specifications but are identifiably different from the rest of the population. Several factors could be contributing to these anomalies, including a part’s location on a wafer, its wafer location within a lot, its fabrication time, the test equipment used to obtain its measurements, the presence of defects, or any combination of these. These anomalies typically manifest themselves in the Tkept hyperspace as passing or failing parts in subspaces predominantly covered by parts of the opposite kind. Outlying and drifting parts can therefore lead to an erroneous derivation of Fi(Tkept), which in turn can cause a higher misprediction rate, especially for failing parts. Therefore, it’s important to “scrub” the collected test measurement data to eliminate these anomalies. We propose three scrubbing steps: ■ ■ ■
principal component analysis (PCA), outlier elimination, and drift reduction.
Principal component analysis For the data we examined, we observed that combinations of Tkept measurements can sometimes show increased correlation to yi, the pass/fail outcomes of a test ti ∈ Tred. Using combinations of Tkept measurements for decision tree modeling, therefore, could lead to more accurate subspace partitions and might be better suited for deriving Fi(Tkept). We use PCA to identify combinations of dimensions (tests).9 PCA derives relevant, linear com-
November–December 2006
Measurement t´2
Data scrubbing
(b)
Measurement t´1
Figure 4. Example training data distribution for passing hypercubes with a large amount of white space (cube A), which can lead to misprediction of future failing parts (circled triangle in cube A) (a); and elimination of white space using principal component analysis (PCA) to derive a new Tkept hyperspace (b).
binations of tests from Tkept using the training data to describe an orthonormal Tkept hyperspace. In other words, PCA potentially realigns the training data distribution along the derived orthonormal axes in the Tkept hyperspace. This is beneficial for decision tree modeling because typical partitions in a decision tree are orthogonal to a measurement axis in the Tkept hyperspace. When we apply decision tree partitioning to the orthonormal hyperspace derived using PCA, the hypercube boundaries are usually parallel to the population spreads. So, the resulting hypercubes might now contain a significantly reduced amount of white space. As with hypercube collapsing, we believe that these white spaces probably belong to failing subspaces and therefore should not be in passing hypercubes. According to this rationale, Fi(Tkept) obtained from a PCA-based orthonormal description of Tkept hyperspace would probably be more accurate in predicting future pass/fail outcomes. Figure 4a shows an example of a training data distri-
457
Process Variation and Stochastic Design and Test
Measurement t2
Outliers leading to spurious passing hypercubes
Measurement t1
Measurement t2
(a)
(b)
Measurement t1
Figure 5. Part distribution example showing the presence of outliers leading to spurious passing hypercubes in failing subspaces (a), and
bution but still satisfy all specifications. As mentioned earlier, the presence of outliers can corrupt the derivation of Fi(Tkept). Specifically, an outlier that passes all tests can reside in a subspace that should be classified as failing in order to minimize prediction error. What results, however, because of one or more outliers, is the creation of a passing hypercube within a failing subspace. Because outliers are inherently unlikely, the passing hypercube will lead to the misprediction of future failing parts. We classify a passing datum as an outlier on the basis of its Euclidean distance from its nearest passing neighbors. If this distance is greater than a preset threshold, we identify the part as an outlier and eliminate it from the training data. Figure 5a shows a part distribution example with circles and triangles representing passing and failing parts, respectively. The decision tree partitioning of the distribution in Figure 5a results in spurious passing hypercubes (shown as shaded areas) in the failing subspaces because of the presence of outlying passing parts. The decision tree model derived from the part distribution shown in Figure 5a will therefore mispredict future failing parts residing in any of these failing subspaces. Eliminating these outlying passing parts removes these erroneous hypercubes from the decision tree model and results in a more accurate Fi(Tkept). Figure 5b illustrates how removal of these outlying passing parts eliminates the spurious passing hypercubes.
elimination of outliers leading to removal of these spurious hypercubes (b).
bution for a Tkept hyperspace consisting of two tests, t1 and t2. Once again, the circles represent passing parts, and the triangles represent failing parts. Figure 4a also shows a possible partitioning of the hyperspace. Passing hypercube A has a significant amount of white space that contains no training part. As a result, the decision tree model will erroneously classify as passing a future failing part (shown as a circled triangle) residing in this empty region. Figure 4b illustrates the partitioning of the same data distribution in a Tkept = {t′1, t′2 } hyperspace derived using PCA. The resulting passing hypercubes are minimal and contain very little white space. With this new partitioning, the decision tree model correctly identifies the future failing part.
Outlier elimination Outliers are parts in the Tkept hyperspace with test measurements that significantly deviate from the distri-
458
Drift reduction Another potential source of error in the derivation of Fi(Tkept) is wafer or lot drift, in which the part distribution’s location in the Tkept hyperspace shifts from wafer to wafer or lot to lot. Consequently, different wafer or lot distributions that ideally should have overlapped to describe homogeneous (passing or failing) subspaces are now skewed because of drift, resulting in erroneous overlap of passing and failing parts. Thus, drift can cause misclassification of future parts. Figure 6a shows distributions for two example groups of test data, indicated by solid and hollow markers. Again, circles in each group represent passing parts, and triangles represent failing parts. The shaded areas show overlap of passing and failing hypercubes of trees derived from the two groups of data. Therefore, a decision tree model derived from one group will make mispredictions with respect to the other group. We minimize drift by measuring the Euclidean distance between the two distributions’ medians (a representation of the distribution centers) and then
IEEE Design & Test of Computers
Test compaction experiment We conducted a test compaction experiment using a commercially available MEMS accelerometer11—an integrated transducer that converts acceleration to an electrical signal. The testing process for an accelerometer involves not only electrical tests but also mechanical tests (that is, tests aimed at evaluating the accelerometer’s mechanical components) at room, cold, and hot temperatures.12 These mechanical tests are far more expensive than their all-electrical counterparts. Therefore, we can avoid significant cost if we can predict the outcomes of these mechanical tests using only the electrical-test measurements. We applied our statistical test compaction methodology to test measurements for more than 70,000 accelerometers. Test engineers collected these measurements at various times over a nine-month period. We investigated the possible elimination of the mechanical tests at both cold and hot temperatures. We used 80% of the accelerometers for training, and all of them for validation. Using fivefold cross validation, we measured the prediction accuracy after eliminating each mechanical test. For this type of validation, we derived five different training data sets from the test measurements. Any pair of training data sets differed from each other for 20% of the parts. We then used each training data set to derive a decision tree model, which we eval-
November–December 2006
Measurement t2
Drift between the two groups of test data
Subspaces with mispredictions
Measurement t1 (a)
Measurement t2
subtracting this value from the test measurements of each part in the distribution being predicted. This is simply the linear translation of a future distribution to the one used to derive the decision tree. Using a median as a central measurement for a part distribution reduces sensitivity to outliers. Figure 6b shows the result of drift reduction for the two distributions in Figure 6a. Specifically, we use the difference between the two groups’ medians to shift the solid-marked data onto the hollow-marked data. This shift eliminates the overlap of passing and failing parts from the two distributions and thus improves the prediction accuracy of the decision tree model derived from the hollow-marked data. Applying these data-scrubbing techniques to a commercial chip, however, will likely require several modifications to the standard test flow. For example, pass/fail prediction for a fabricated part is possible only after collecting the test measurements from all chips in a lot or wafer. We are currently exploring the issues related to making data scrubbing feasible in an industry test environment, and the work of Turakhia et al. could provide a viable solution.10
Measurement t1 (b) Figure 6. Drift between the distributions of two example groups of test data with Tkept = {t1, t2} (a), and elimination of the misprediction error through drift reduction (b). Shaded regions in (a) show places where parts are mispredicted.
uated for the amount of misprediction upon elimination of a test. The average level of misprediction for the five training data sets characterized the classifier’s quality for the possible elimination of the test. We observed that the test measurement data used in this experiment included outliers and lot-to-lot drift, so we scrubbed the data to eliminate the effect of drift. In addition, we treated the training data to remove passing outliers. Finally, we used guard banding to reduce misprediction for tests whose elimination resulted in unacceptable errors. Using custom software developed at Carnegie Mellon University, we constructed a tree in just a few hours, using a data set of more than 50,000 parts. Using this tree for classification took only a few seconds for all of the tens of thousands of parts predicted.
459
Process Variation and Stochastic Design and Test
Table 1. Fivefold cross-validation percentages of passing
Table 2. Fivefold cross-validation percentages of passing
and failing parts mispredicted and their averages when
and failing parts mispredicted and their averages when
we eliminate the cold mechanical test.
we eliminate the hot mechanical test.
Failing parts
Passing parts
Run
mispredicted (%)
mispredicted (%)
Run
1
0.00
0.06
1
9.34
0.72
2
1.54
0.08
2
8.79
0.65
3
0.30
0.06
3
4.40
0.75
4
0.00
0.07
4
7.69
0.71
5
0.00
0.07
5
4.40
0.82
0.36
0.07
6.92
0.73
Average
Failing parts
Passing parts
mispredicted (%)
mispredicted (%)
Average
the hot mechanical test. When we eliminated the 56.1% 0.89% cold mechanical test, Cold 34.6% the average misprediction Hot error was only 0.36% for 98 failing parts and 0.07% 16.3% for passing parts; when we eliminated the hot mechanical test, the resulting error was 6.92% and 96 0.73%, respectively, for the failing and passing parts. Although the percent4.3% ages of mispredictions for 94 the mechanical test outcomes are extremely low, we can use specification guard banding to further 92 0 10 20 30 40 50 60 70 80 reduce these values. In fact, we can easily achieve Guard-banded parts (%) zero defect escape for the cold mechanical test. Figure 7. Worst-case receiver operating characteristic (ROC) plots of percentages of Figure 7 shows receiver failing parts correctly identified versus percentage of guard-banded parts for a operating characteristic commercial accelerometer when cold (solid line) and hot (dotted line) (ROC) plots for the worstmicromechanical tests are eliminated. case fivefold cross-validation runs for cold (solid We conducted separate test compaction experiments line) and hot (dotted line) mechanical tests. Specifically, to analyze the possible elimination of the mechanical tests Figure 7 plots the percentages of correct predictions for at both cold (–40°C) and hot (80°C) temperatures. We failing parts against the percentage of parts placed in the included all the electrical tests at each temperature in Tkept guard band. (Note that the percentages reported along to derive the decision tree model for the outcome of the the y-axis in Figure 7 are not the same as DPM—number mechanical test at that temperature. Table 1 gives the five- of defective parts per million units shipped—but rather fold cross-validation prediction results when we elimi- they are the percentages of failing parts mispredicted.) nated the cold mechanical test. Table 2 gives the fivefold These plots demonstrate that it’s possible to improve the cross-validation prediction results when we eliminated accuracy of Fi(Tkept). For example, we could probably 8.1%
78.4%
Failing parts correctly identified (%)
100
460
IEEE Design & Test of Computers
eliminate the cold mechanical test, because few parts require guard banding to ensure that no defective parts escape. To illustrate the potential cost savings from using our test compaction methodology, we developed a simple model for the reduction in test cost per shipped part, ΔC:
work, and we thank Freescale employees Teresa Maudie, Rick Neilsen, Ray Roop, and Brooks Scofield for their insightful description of the test data.
References 1. V.N. Vapnik, Statistical Learning Theory, WileyInterScience, New York, 1998.
ΔC = (CE + CM)/Y – [CE + (CDE × DE) + (CM × GB)]/(Y – ΔY)
2. A.K. Jain, J. Mao, and K.M. Mohiuddin, “Artificial Neural Networks: A Tutorial,” IEEE J. Computers, vol. 29, no. 3, Mar. 1996, pp. 31-44.
where CE and CM are the cost of applying the electrical and mechanical tests to each manufactured part, CDE is the cost of mispredicting a defective part, and Y is the process yield. ΔY and DE are the fractions of good and failing parts, respectively, that are mispredicted because of the elimination of a mechanical test, and GB is the fraction of parts guard-banded (meaning they require application of the mechanical test). Assuming Y = 90%, CM = $5, CE = $1, and CDE = $1,000, the model simplifies to
3. L. Rokach and O. Maimon, “Top-Down Induction of Decision Trees Classifiers—A Survey,” IEEE Trans.
Systems, Man, and Cybernetics—Part C: Applications and Reviews, vol. 35, no. 4, Nov. 2005, pp. 476-487. 4. S. Biswas et al., “Specification Test Compaction for Analog Circuits and MEMS,” Proc. Design, Automation and
Test in Europe Conf. (DATE 05), IEEE CS Press, 2005, pp. 164-169. 5. H.G.D. Stratigopoulos and Y. Makris, “Bridging the Accuracy of Functional and Machine-Learning-Based Mixed-
ΔC = 6.67 – [1 + (1000 × DE) + (5 × GB)]/(0.9 – ΔY)
Signal Testing,” Proc. IEEE VLSI Test Symp. (VTS 06), IEEE CS Press, 2006, pp. 395-400.
Substituting DE = 0.36% and ΔY = 0.07% from Figure 7, the cost reduction from eliminating the cold mechanical test with no guard banding is $5.16 per shipped part. If, however, no defect escape is tolerable, we must guard-band 8.1% of the manufactured parts. Assuming we apply both the cold mechanical and electrical tests to these guard-banded parts, the cost reduction is still significant: $5.11 per shipped part. Either way, we eliminate more than 76% of the cold test’s cost.
6. B. DasGupta, H.T. Siegelman, and E. Sontag, “On the Complexity of Training Neural Networks with Continuous Activation Functions,” IEEE Trans. Neural Networks, vol. 6, no. 6, Nov. 1995, pp. 1490-1504. 7. J.K. Martin and D.S. Hirschberg, The Time Complexity of
Decision Tree Induction, tech. report 95-27, Dept. of Information and Computer Science, Univ. of California, Irvine, 1995. 8. R. Voorakaranam et al., “Production Deployment of a Fast Transient Testing Methodology for Analog Circuits: Case Study and Results,” Proc. Int’l Test Conf. (ITC 03),
THUS, OUR PROPOSED METHODOLOGY can eliminate
an expensive mechanical test for a commercially available accelerometer with little error. Moreover, it’s possible to completely eliminate the error (for failing parts) using specification guard banding. But the same result could not be achieved for the equivalent mechanical test executed at an elevated temperature. Techniques such as specification guard banding and drift removal can reduce error, but more research is needed. More importantly, techniques are needed for incorporating this and similar methodologies into a production test flow. ■
IEEE CS Press, 2003, pp. 1174-1181. 9. M.E. Wall, A. Rechsteiner, and L.M. Rocha, “Singular Value Decomposition and Principal Component Analysis,” A Practical Approach to Microarray Data Analysis, D.P. Berrar, W. Dubitzky, and M. Granzow, eds., Kluwer Academic Publishers, 2003, pp. 91-109. 10. R.P. Turakhia et al., “Changing Test and Data Modeling Requirements for Screening Latent Defects as Statistical Outliers,” IEEE Design & Test, vol. 23, no. 2, Mar.-Apr. 2006, pp. 100-109. 11. S.D. Senturia, “A Capacitive Accelerometer,” Microsystem
Design, Kluwer Academic Publishers, 2003, pp. 497-530.
Acknowledgments We acknowledge the use of Freescale Semiconductor accelerometer test data in this research
November–December 2006
12. T. Maudie et al., “MEMS Manufacturing Testing: An Accelerometer Case Study,” Proc. Int’l Test Conf. (ITC 03), IEEE CS Press, 2003, pp. 843-849.
461
Process Variation and Stochastic Design and Test
F E AT U R I N G
IN 2007 • Healthcare • Mining a Sensor-Rich World • Urban Computing • Security & Privacy IEEE Pervasive Computing delivers the latest peer-reviewed developments in pervasive, mobile, and ubiquitous computing to developers, researchers, and educators who want to keep abreast of rapid technology change. With content that’s accessible and useful today, this publication acts as a catalyst for progress in this emerging field, bringing
Sounil Biswas is a PhD candidate in the Department of Electrical and Computer Engineering at Carnegie Mellon University. His research interests include test of integrated, heterogeneous systems; statistical analysis of test data; and defect characterization. Biswas has a BTech in electrical engineering from the Indian Institute of Technology, Kanpur, and an MS in electrical and computer engineering from Carnegie Mellon University.
Ronald D. (Shawn) Blanton is a professor in the Department of Electrical and Computer Engineering at Carnegie Mellon University, where he is an associate director of the Center for Silicon System Implementation (CSSI). His research interests include test and diagnosis of integrated, heterogeneous systems. Blanton has a BS in engineering from Calvin College, an MS in electrical engineering from the University of Arizona, and a PhD in computer science and engineering from the University of Michigan, Ann Arbor. He is a member of the ACM and a senior member of the IEEE.
together the leading experts in such areas as • Hardware technologies • Software infrastructure • Sensing and interaction with the physical world
Subscribe Now!
Direct questions and comments about this article to Shawn Blanton, 2109 Hamerschlag Hall, Dept. of Electrical and Computer Engineering, Carnegie Mellon University, 5000 Forbes Ave., Pittsburgh, PA 15217;
[email protected].
• Graceful integration of human users • Systems considerations, including scalability, security, and privacy
For further information on this or any other computing topic, visit our Digital Library at http://www.computer.org/ publications/dlib.
VISIT
www.computer.org/pervasive/subscribe.htm
462
IEEE Design & Test of Computers