Selection Methods of Control Samples:

A Comparison of Two Matching Methodologies

 

 

 

Jong Hwan Yi

California State University, Los Angeles

School of Business and Economics

5151 State University Drive

Los Angeles, CA 90032

 

Tel: 323-343-2877

Fax: 323-343-2885

Email:

 

 

 

 

Selection Methods of Control Samples:

A Comparison of Two Matching Methodologies

 

 

 

Abstract

 

To measure abnormal stock returns of a sample of firms in an event study, we often use matching firm adjusted returns where returns of the control firms are subtracted from the raw returns of the sample firms.  In most financial studies, the control firms are selected by matching industry and size (the I/S method).  That is, for each sample firm, a matching firm with the closest market capitalization within the same 3-digit Standard Industry Classification (SIC) code is selected.  In this study, an alternative control firm selection method based on earnings per share (the EPS method) is compared to the traditional method.  The EPS method matches each sample firm with a control firm that has the same EPS for a given fiscal year.  While the mean matching firm adjusted returns provided by the two methods are close to the expected value of zero, the size of variances of the adjusted returns is somewhat smaller for the I/S method, showing some superiority for the traditional matching method based on industry and size.   

 

 

 

Selection Methods of Control Samples:

A Comparison of Two Matching Methodologies

 

 

Introduction

 

To measure abnormal stock returns of a sample of firms in an event study, we often use matching firm adjusted returns.  A matching firm adjusted return is defined as the return of a sample firm minus the return of its control firm over the same period.  In most financial studies, these control firms are selected by matching the industry and size of the sample firms (the I/S method).  That is, for each sample firm, a matching firm with the closest market capitalization within the same 3-digit Standard Industry Classification (SIC) code is selected.  Since two firms in the same industry are likely to be subject to the same industry conditions, matching industry can isolate any industry-specific factors that affect the stock returns of sample firms.  Similarly, matching firm size attempts to isolate any factors that can affect companies of certain size.  An early empirical study of the size effect is Banz (1981), who reports that small firms tend to have higher stock performance than larger firms. 

There are numerous empirical studies that control for industry and size effects including Ritter (1991) and Spiess and Affleck-Graves (1995).  In these studies, matching firms are those that have the closest firm size (usually proxied by the market capitalization) within the same industry.  Other studies, such as Loughran and Ritter (1995) and Brav and Gompers (1995) control for the size of the firm only.

This study tests an alternative way of selecting matching firms for a sample of firms: matching based on earnings per share (the EPS method).  While the conventional method of matching based on industry and size tries to control for possible effects specific to certain industries and size, matching based on earnings per share tries to control for the current level of EPS in assessing abnormal returns of a stock relative to another.  To the extent to which future stock returns are correlated with current EPS, controlling for current EPS would isolate the EPS effect that would contribute to any possible abnormal returns.  Some of the studies that study the relationship between earnings and stock returns include Basu (1983), Kim (1997), and Chia, Czernkowski and Loftus (1997).

Obviously, in an event study measuring abnormal stock returns, we would have more confidence in any inferences that we might draw from a better methodology.  The question of, which of the two matching methods is more appropriate, seems to be largely an empirical one.  Thus, we conduct an experiment using randomly selected sample firms with two sets of matching firms based on the two methods described above.  The superior method would be the one that produces a set of matching firms with closest returns to those of the sample firms.  One way to test this is to measure the matching firm adjusted returns (the returns of sample firms minus the returns of matching firms) over various holding periods.  Since the sample firms are randomly selected from the general population, a superior method would be defined as the one that yields matching firms with mean returns closest to zero and the smallest variances.

The remaining sections are as follows.  In section 2, we describe the selection process of sample firms and matching firms using the two methodologies.  Section 3 presents the analysis and results.  Section 4 summarizes and concludes the paper.

 

Selection of sample firms and matching firms

 

            In our experiment, we first construct a set of sample firms by randomly selecting 500 firms from both the NYSE and NASDAQ CRSP files.  We restrict our population to those that have publicly traded at least 5 years to avoid any initial public offerings, which are known to have both short-run and long-run abnormal returns.  A summary of abnormal short-run returns and long-run abnormal returns of initial public offerings are reported in Smith (1986) and Ritter (1991), respectively.  We also exclude firms with negative earnings for any sample year because they may be under special circumstances, for example, financial distress.  Firms with financial distress may be subject to some abnormal returns and may bias the results of the study.  For each of the 5 years during the 1987-1991 sample period, we randomly choose 100 firms.  We study 5 sample years rather than a single year for robustness of the test.  For the 500 firms in the sample, we construct two sets of matching firms using the two different methods.  The matching firms are also selected from the same population as for the sample firms without replacement.

The first set of firms is selected using the traditional method of matching by industry and size (the I/S method).  Our selection process using the I/S method discussed here is similar to the one used in Ritter (1991).  For industry, we use the Standard Industry Classification (SIC) code.  The market capitalization of a firm, which proxies for the firm size, is defined as the market price per share times the number of shares outstanding.  For each sample firm, we select a matching firm that has the closest market capitalization within the same 3-digit SIC code.  For example, a sample firm in 1990 is matched by a firm in the same 3-digit SIC code that has the closest market capitalization at the end of the same year 1990. 

From the 500 pairs of sample and matching firms, we drop the firms (either sample or matching firms) that are delisted from CRSP return files before the three-year anniversary (the longest holding period examined in this study) since the first date of return calculation.  This avoids any selection bias that may result from including firms that would be merged, bankrupt or liquidated.  The final numbers of observations are 394 pairs of firms for the I/S method.

Table 1 compares the market capitalizations of the samples firms with those of the matching firms selected by I/S method.  The Market capitalization is defined as the stock price at the first calendar day of a year times the number of shares outstanding.  The mean (median) values are 150.9 (66.9) million dollars for the sample firms and 133.2 (62.7) million dollars for the matching firms.  The slightly greater numbers for the sample firms are seen in every sample year except 1989.  The difference, however, is not sizable and does not seem to cause any significant bias in the results.

 

Table 1

Market capitalizations of sample firms and matching firms based on industry and size

 

The sample firms are randomly selected from CRSP tapes between 1987 and 1991.  The matching firms are selected by using the industry and size (I/S) method.  Market capitalization is the stock price at the first calendar day in a year times the number of shares outstanding.

 

 

 

Sample Firms

Matching Firms (I/S)

Year

Number

of Pairs

Mean (median) Market Cap.

($ millions)

Mean (median) Market Cap.

($ millions)

1987

81

145.5 (69.9)

131.9 (54.8)

1988

75

145.3 (60.4)

123.4 (48.4)

 1989

88

96.8 (67.8)

102.4 (74.2)

1990

78

174.2 (70.2)

167.2 (68.5)

1991

72

160.8 (70.0)

145.7 (69.1)

Total

394

150.9 (66.9)

133.2 (62.7)

 

 

We choose the second set of control firms using the EPS matching method.  That is, for each sample firm, we select the matching firm that has the same EPS for the same fiscal year.  Since there were many potential matching firms that had the same EPS for a given sample firm, we randomly selected one with the same EPS.  After excluding the firms that have been delisted before three years, there were 354 pairs of sample and matching firms with the same EPS.  The sample is smaller using the EPS method, compared to 394 for the I/S method, mainly because a larger number of matching firms selected by this method was delisted before three years.  Another reason for the different sample size is that in the I/S method, there were more cases where both the sample firm and the matching firm were delisted before the three years. 

Table 2 shows the mean and median EPS for the sample firms and the matching firms selected by using the EPS method.  Since each and every sample firm was matched by a firm that had the same EPS, both the mean and the median values are identical for both set of firms at 0.67 and 0.48 dollars, respectively.

 

Table 2

Descriptive statistics of sample firms and matching firms based on earnings per share

 

The sample firms are randomly selected from CRSP tapes between 1987 and 1991.  The matching firms are selected by using the earnings per share (EPS) method.

 

 

 

Sample Firms

Matching Firms (EPS)

Year

Number

Of Pairs

Mean (median)

EPS ($)

Mean (median)

EPS ($)

1987

78

0.67 (0.48)

0.67 (0.48)

1988

74

0.62 (0.45)

0.62 (0.45)

 1989

75

0.64 (0.54)

0.64 (0.54)

 1990

68

0.71 (0.47)

0.71 (0.47)

1991

59

0.67 (0.50)

0.67 (0.50)

Total

354

0.67 (0.48)

0.67 (0.48)

 

 

Analysis and Results

 

In order to compare the two methods, we use a simplified version of the simulation event study used in Brown and Warner (1980).  Brown and Warner were interested in finding the relative power of different event study methodologies.  Their methodology involved taking a random sample of stocks and introducing artificial abnormal performance.

To compare the two methods, we construct two series of matching firm adjusted holding period returns based on dividend and split-adjusted stock prices.  The matching adjusted holding period returns are defined as the holding period return of a sample firm minus the holding period return of its matching firm.  The four holding periods we examine are 6 month, one year, two years and three years beginning the first trading day of each calendar year.  Those are

 

ARET6 = Rsample,6 – Rmatch,6

ARET12 = Rsample,12 – Rmatch,12

ARET24 = Rsample,24 – Rmatch,24

ARET36 = Rsample,36 – Rmatch,36

 

where Rsample,j  is the j-month holding period returns of a sample firm

           Rmatch,j is the j-month holding period returns of its matching firm

and     ARETj is the j-month matching firm adjusted return.

 

            Because this experiment involves a random selection of seasoned stocks, we would expect to find no abnormal performance of our sample firms (zero mean adjusted returns) if our matching schemes are appropriate.  Our objective is to find the method that provides a better fit to the returns of our sample firms, that is we are searching for the method whose adjusted return series have a mean of zero and a smaller variance of the mean.

            Table 3 presents the first comparison of the two methods.  For holding periods of 6, 12, 24, and 36 months, we report the mean and median matching firm adjusted returns using of the two methods.  While the mean returns provided by the two methods are close to the expected value of zero, it appears that for each of the four holding periods the I/S method produces mean returns closer to zero in absolute value.  For the I/S method, the mean returns for the four periods are 0.0048, 0.0042, -0.0143 and –0.0003, respectively.  The mean returns for the EPS method are 0.0154, 0.0111, -0.0279 and –0.0096.  In terms of the median adjusted returns, each of the methods yields lower absolute values in two of the four holding periods.  In any case, all the mean and median returns are not significantly different from zero, and thus neither method can be chosen as the superior one based on them.

 

Table 3

Comparison of mean (median) matching firm adjusted returns and their variances of the two matching methods: industry and size versus earnings per share

 

This table reports the mean (median) matching firm adjusted returns and their variances of a s ample of seasoned firms using two different methods of selecting the control sample.  The industry and size method selects a matching firm by choosing the firm with the closest market capitalization among the firms that share the same 3-digit SIC code.  The earnings per share method selects a matching firm based on the EPS of a fiscal year.  ARET6, ARET12, ARET 24 and ARET36 are 6-month, one-year, two-year and three-year matching firm adjusted holding period returns, respectively.  F-statistic is obtained by dividing the larger variance by the smaller variance.  This assumes that the two samples are independent.

 

Abnormal

Returns

Industry and size (n=394)

Earnings per share (n=354)

F-statistic

 

Mean

(median)

Variance of mean

Mean

(median)

Variance of

mean

 

ARET6

0.0048 (0.0084)

0.07456

0.0154 (0.0134)

0.05849

1.27

ARET12

0.0042 (0.0091)

0.07022

0.0111 (0.0069)

0.04823

1.46*

ARET24

-0.0143 (0.0018)

0.05688

-0.0279 (-0.0133)

0.06678

1.17

ARET36

-0.0003 (-0.0048)

0.04130

-0.0096 (-0.0001)

0.05220

1.26

 

* The two variances are significantly different from each other at the 5% level.

 

We now turn to the variances of the mean adjusted returns presented in Table 3.  An examination of the variances shows that for two of the four periods (6 and 12 month), the EPS method yields smaller variance.  However, as is the case with the mean returns, the variances are close to each other.  To formally test the difference of variances, F-statistics can be computed by dividing the larger variance by the lower variance.  This test shows that, except for the 12-month period in which the EPS method has a smaller variance, there is no significant difference between the variances of the mean returns.  Moreover, since the returns produced by the two methods are not likely to be independent from each other, the F-test may be biased.  Therefore, the overall evidence presented in Table 3 does not suggest that either matching method is superior.

            Another way to compare the predictive ability of the two matching methods is to examine the performance of the two methods at the individual firm level.  Comparing the performance of the two methods for each of our sample firms, we should prefer the method that more frequently produces smaller (in absolute value) adjusted returns.  Again, this criterion is based on our expectation for the adjusted returns to be zero. 

For the comparison described above, we would need the same sample size for the two methods.  Thus we use the 354 sample firms with matching firms selected by both the I/S and the EPS methods.  All these firms have complete return data for three years examined.  For each method, we compute the adjusted returns over the holding periods of 6, 12, 24 and 36 months.  Then, for each firm at each holding period, we observe which of the two methods produces a lower (in absolute value) adjusted return.  The method that more frequently produces lower adjusted returns would be the superior method.  If the two methods are equally good ways to choose the control firms, we would expect that each of the two methods would have smaller abnormal returns for 50% of the firms.

The results are presented in Table 4.  The results show that for each of the four holding periods, the I/S method has lower adjusted returns more than 50% of the time.  The percentage ranges from a low of 56.2% (36 month holding period) to a high of 64.4% (24 month holding period).  If we assume that the returns produced by the two methods are independent from each other, a simple binomial test can be used to see if these percentages are significantly different from the 50%.  The test shows that the difference is significant at the 5% level for each holding period.  Although to the extent of any cross-sectional correlation in the security returns, the tests may be biased, the overall results suggest that the traditional matching method based on industry and size is superior.

 

 

Table 4

Comparison of two matching methods at individual firm level

 

This table reports the frequency for which the industry and size matching method produces a smaller adjusted return (in absolute value) than the EPS matching method.  Adj (I/S) is the absolute value of an individual firm’s adjusted return when matching on industry and size is used.  Adj (EPS) is the absolute value of an individual firm’s adjusted return when matching on earnings per share is used.  A total of 354 sample firms are used in the experiment.

 

Holding Period

Number of firms

in the sample

Number of firms with

Adj (I/S) < Adj (EPS)

% of firms with

Adj (I/S) < Adj (EPS)

6-month

354

219

61.9*

One-year

354

206

58.2*

Two-year

354

228

64.4*

Three-year

354

199

56.2*

 

*  Significantly higher than 50% by binomial test at 5% level.

 

Summary and Conclusion

 

In this study, we run an experiment to compare two selection methods of control sample.  To measure abnormal returns of a sample of stocks in an event study, we often use matching firm adjusted returns where returns of the control firms are subtracted from the raw returns of the sample firms.  In most financial studies, the control firms are selected by matching industry and size (the I/S method).  An alternative matching method based on earnings per share (the EPS method) is compared to the traditional method.  The EPS method matches each sample firm with a control firm that has the same EPS for the same fiscal year.  While the mean matching firm adjusted returns provided by the two methods are close to the expected value of zero, the size of variances of adjusted returns is somewhat smaller for the I/S method.  Also, at the individual firm level, the I/S method produced a larger number of matching firms whose holding period returns were closer to the returns of the sample firms, showing some superiority for the traditional matching method based on industry and size.

References

 

Banz, Rolf, 1981, The Relationship between Return and Market Value of Common Stocks, Journal of Financial Economics 9, 3-18.

 

Basu, Sanjoy, 1983, The Relationship between Earnings Yield, Market Value, and Return for NYSE Common Stocks: Further Evidence, Journal of Financial Economics 12, 129-156.

 

Brav, Alon and Paul Gompers, 1997, Myth or Reality?  The Long-Run Underperformance of Initial Public Offerings: Evidence from Venture and Nonventure Capital-backed Companies, Journal of Finance 52, 1791-1821.

 

Brown, Stephen and Jerold Warner, 1980, Measuring Security Price Performance, Journal of Finance 8, 205-258.

 

Chia Y, R. Czernkowski and J. Loftus, 1997, The Association of Aggregate and Disaggregatd Earnings with Annual Stock Returns, Accounting and Finance 37, 111-128. 

 

Kim, Dongcheol, 1977, A Reexamination of Firm Size, Book-to-Market, and Earnings Price in the Cross-Section of Expected Stock Returns, Journal of Financial and Quantitative Analysis 32, 463-489.

 

Loughran, Tim and Jay Ritter, 1995, The New Issues Puzzle, Journal of Finance 50, 23-51.

 

Ritter, Jay, 1991, The Long-Run Performance of Initial Public Offerings, Journal of Finance 46, 3-27.

 

Smith, Clifford Jr., 1986, Investment Banking and the Capital Acquisition Process, Journal of Financial Economics 15, 3-30.

 

Spiess, K and J. Affleck-Graves, 1995, The Long-Run Performance following Seasoned Equity Offerings, Journal of Financial Economics 38, 243-267.