Statistics with STATA UPDATED VERSION 9

Item

Title
Statistics
with STATA UPDATED VERSION 9
extracted text
I

Statistics
with

Updated for Version 9

I

Lawrence C. Hamilton
University ofNew Hampshire

I
I

i

ANOTHER QUALITY
USED BOOK


•.



-

.

-

Australia • Brazi __ _______________ _________ _____________
Ur..

ItrtKHi
THOMSON
-------------*------------BROOKS/COLE

Statistics with Stata: Updated for Version 9
Lawrence C Hamilton
Publisher: Curt Hinrichs
Senior Assistant Editor: Ann Dav
Editorial Assistant: Daniel Geller
Technology Project Manager: Fiona Chong
Marketing Manager: Joe Rogove
Marketing Assistant: Brian Smith
Executive Marketing Communications Manager:
Darlene Amidon-Brent

© 2006 Duxbury, an imprint of Thomson Brooks. Cole,
a part of The Thomson Corporation. Thomson, the Star
logo, and Brooks/Cole are trademarks used herein
under license.
ALL KIGHTS RESERVED. No part of tliis work cox ered
b\ the copyright hereon may be reproduced or used in
any form or by any means—graphic, electronic, or
mechanical, including photocopying, recording, taping,
web distribution, information storage and retriex al
sy stems, or in any other manner—without the written
permission of tire publisher.

Printed in Canada
1 2 3 4 5 6 7 09 08 07 06 05

Project Manager, Editorial Production: Kelsey McGee
Creative Director: Rob Hugel
Art Director: Lee Friedman
Print.Buyer: Darlene Suruki
Permissions Editor: Kielx- Sisk
Cover Designer: Denise Davidson/Simple Design
Cover Image: ©Imtek Imagineering/Masterfile
Cover Printing, Printing & Binding: Webcom Limited
Thomson Higher Education
10 Davis Drive

Belmont, CA 94002-3098
USA
Asia (including India)
Thomson Learning
5 Shen ton Way
#01-01 UIC Building
Singapore 068808

Australia/New Zealand
Thomson Learning Australia
102 Dodds Street
Southbank, Victoria 3006
Australia
Canada

For more information about our nrndi

n Nelson
rhmount Road
Ontario MIK 5G4

another quality
USED BOOI^h
I■

pe/Middle East/Africa
i Learning
born House
iford Row
VC1R 4LR
ngdom

I
Community Health Cell
Library and Information Centre

ff 007, “Srinivasa Nilaya”
Jakkasandra 1st Main,
1st Block, Koramangala,
BANGALORE - 560 034.
Phone : 553 15 18 / 552 53 72
e-mail : chc@sochara.org

i

Contents
Preface

ix

1

Stata and Stata Resources
A Typographical Note ..................
An Example Stata Session ..............
Stata’s Documentation and Help Files
Searching for Information..................
Stata Corporation
Statalist
The Stata Journal
Books Using Stata

1
. I
. 2
. 7
. 8
9
10
10
ll

2

Data Management
Example Commands...........................................
Creating a New Dataset ...................................................
Specifying Subsets of the Data: in and if Qualifiers
Generating and Replacing Variables ..............................
Using Functions ...............................................................
Converting between Numeric and String Formats..........
Creating New Categorical and Ordinal Variables ..........
Using Explicit Subscripts with Variables........................
Importing Data from Other Programs..............................
Combining Two or More Stata Files ................................
Transposing. Reshaping, or Collapsing Data..................
Weighting Observations.....................................................
Creating Random Data and Random Samples
Writing Programs for Data Management...........................
Managing Memory.............................................................

12
. 13
. 15
. 19
. 23
. 27
. 32
. 35
. 38
. 39
. 42
. 47
. 54
. 56
. 60
. 61

3

Graphs
Example Commands . . . .
Histograms
Scatterplots
Line Plots
Connected-Line Plots . . .
Other Twoway Plot Types
Box Plots
Pie Charts
Bar Charts

64
65
67
72
77
83
84
90
92
94

vi

Statistics with Stata

Dot Plots..............
Symmetry and Quantile Plots ....
Quality Control Graphs
Adding Text to Graphs
Overlaying Multiple Twoway Plots
Graphing with Do-Files ................
Retrieving and Combining Graphs

4

Summary Statistics and Tables
Example Commands.....................................
Summary Statistics for Measurement Variables
Exploratory Data Analysis
Normality Tests and Transformations
Frequency Tables and Two-Way Cross-Tabulations
Multiple Tables and Multi-Way Cross-Tabulations
Tables of Means, Medians, and Other Summary Statistics
Using Frequency Weights

5

ANOVA and Other Comparison Methods
Example Commands
One-Sample Tests
Two-Sample Tests .................................
One-Way Analysis of Variance (ANOVA)
Two- and TV-Way Analysis of Variance .. .
Analysis of Covariance (ANCOVA)
Predicted Values and Error-Bar Charts

6

Linear Regression Analysis
Example Commands
The Regression Table
Multiple Regression .....................................
Predicted Values and Residuals
Basic Graphs for Regression
Correlations........................................
Hypothesis Tests...........................................
Dummy Variables
Automatic Categorical-Variable Indicators and Interactions
Stepwise Regression
Polynomial Regression
Panel Data
. .

. 99
100
105
109
110
115
116
120
120
121
124
126
130
133
136
138

141
142
143
146
149
152
153
155
159
159
162
164
165
168
171
175
176
183
186
188
191

£

Contents

vii

7

Regression Diagnostics
Multicollinearity
........................
Diagnostic
Case Statistics
Example Commands
Diagnostic
........................
SAT Score Plots
Regression.
Revisited

8

Fitting Curves
Example Commands
Band Regression
Lowess Smoothing
Regression with Transformed Variables - I
Regression with Transformed Variables - 2
Conditional Effect Plots
Nonlinear Regression - I
Nonlinear Regression - 2

215
215
217
219
223
227
230
232
235

9

Robust Regression
Example Commands
Regression with Ideal Data
Y Outliers
XOutliers (Leverage)
Asymmetrical Error Distributions
Robust Analysis of Variance
Further rreg and qreg Applications
Robust Estimates of Variance— I
Robust Estimates of Variance — 2

239
240
240
243
246
248
249
255
256
258

10 Logistic Regression
Example Commands
Space Shuttle Data
Using Logistic Regression
Conditional Effect Plots
Diagnostic Statistics and Plots
Logistic Regression with Ordered-Category v
Multinomial Logistic Regression

262
263
265
270
273
274
278
280

11 Survival and Event-Count Models
Example Commands
Survival-Time Data
Count-Time Data
Kaplan-Meier Survivor Functions .
Cox Proportional Hazard Models ..
Exponential and Weibull Regression
Poisson Regression

288
289
291
293
295
299
305
309
313

Generalized Linear Models

196
196
198
200
205
210

.

—— mi

via

Statistics with Stata
i

12 Principal Components, Factor, and Cluster Analysis

Example Commands.......................................
Principal Components.......................................
Rotation
Factor Scores........................................
Principal Factoring.....................................
Maximum-Likelihood Factorina
Cluster Analysis — I ......................
Cluster Analysis — 2 ......................
13 Time Series Analysis

Example Commands..........
Smoothing
Further Time Plot Examples
Lags, Leads, and Differences
Correlograms.......................
ARIMA Models .................
14 Introduction to Programming

Basic Concepts and Tools........................
Example Program: Moving Autocorrelation
Ado-File
7........................
Help File
Matrix Algebra
Bootstrapping
Monte Carlo Simulation
References
Index

. 318
. . 319
.. 320
.. 322
.. 323
. . 326
.. 327
.. 329
.. 333
339
339
341
346
349
351
354

361
361
369
373
375
378
382
387
395

401

'7

Stata and Stata Resources

Stata ts a full-featured statistical program for Windows. Macintosh, and Unix computers It
h 'lT 0050 USC "llh SPeCd’ a ,lbrary orPre-P'ogrammed analytical and data-mana«ement
n d
MS'
'lro-ranllllability tllat allows users to invent and add further capabilities as
needed. Most operations can be accomplished either via the pull-down menu system or more
ectly via tv ped commands. Menus help newcomers to learn Stata. and help anyone to apply
an unfamiliar procedure. The consistent, intuitive syntax of Stata commands frees experietKcd
users to work more efficiently, and also makes it straightforward to develop programs foi
mp ex oi repetitious tasks. Menu and command instructions can be mixed as needed durin<>
a Stata session. Extensive help, search, and link features make it easv to look up command
syntax and other information instantly, on the fly.
"
P

After introductory information, we'll begin with an example Stata session to .five you a
sense of the flow of data analysis, and ofhovv analytical results might be used. Later chapters
theTonn "are"3
•\ith°Ut eXplanati°nS' h°Wever’ V™ ca" see how straightforward
e commands are-use filename to retrieve dataset/ttowme, summarize when you
want summary statistics, correlate to get acorrelation matrix, and so forth. Alternatively
same results can be obtained by making choices from the Data or Statistics menus.
Stata users have available a variety of resources to help them learn about Stata and solve
problems at any level of difficulty. These resources come not just from Stata Corporation but

s^aXe'ehc?mmudni • ofLers‘Sections ofthis chapter introduce some keyreso—

technical heln StW ’
documentation: where to phone, fax. write, or e-mail for
undnmA d
7' S'te ('""v staIa com)- which provides many services including
S'Xrn 7"'erSt0 qUen y
qUeSti°nS; the StataliSt Intemet forum; and ,herefereed

A Typographical Note
This book employs several typographical conventions as a visual cue to how words are used:



Commands typed by the user appear in a bold Courier font. When the whole
t“) file"' ’S giVCn’ " S‘artS With 3 Pen°d’ 3S See" in 3 S‘ata ReSultS wind0'v or log
.



list year boats men penalty

Variable or file names within these commands appear in italics to emphasize the
tact that they are arbitrary'and
" ■ "“J not
-:t a fixed part of the command.

1

r~
2

Statistics with Stata



Names of yariables or fifes also appear in italics within the main text to distinguish them
from ordinary words.



Items from Stata’s menus are shown in an Arial font. with successive options separated b\
a dash. For example, we can open an existing dataset by selecting File - Open . and then
finding and clicking on the name of the particular dataset. Note that some common menu
actions can be accomplished cither with text choices from Stata’s top menu bar.
File
Edit
Prefs Data Graphics Statistics User Window
Help
or with the row of icons below these. For example, selecting File - Open is equivalent to
clicking the leftmost icon, an opening file folder:
. One could also accomplish the
same thing by typing a direct command of the form
use filename



Stata output as seen in the Results w indow is showm in a
font allows Stata’s 80-column output to fit within the margins of this book.

The small

Thus, we show the calculation of summary statistics for a variable named penally as
follows:
.

summarize penalty

Variable I

jbs

Wear.

penalty |

10

63

.? 1 :i.

?. 5 9 4.9

These typographic conventions exist only in this book, and not within the Stata program
itself. Stata can display a variety of onscreen fonts, but it docs not use italics in commands.
Once Stata log files have been imported into a word processor, or a results table copied and
pasted, you might want to format them in a Courier font, 10 point or smaller, so that columns
will line up correctly.

In its commands and variable names. Stata is case sensitive. Thus, summarize is a
command, but Summarize and SUMMARIZE are not. Penalty and penalty would be two
different variables.

An Example Stata Session
As a preview showing Stata at work, this section retrieves and analyzes a previous 1\ -created
dataset named lofoten.dta. Jentoft and Kristofferson (1989) originally published these data in
an article about self-management among fishermen on Norway’s arctic Lototen Islands. There
are 10 obset vations (years) and 5 variables, including penalty, a count of how many fishermen
were cited each year tor violating fisheries regulations.
If we might eventually want a record of our session, the best w'ay to prepare for this is by
opening a “log file’ at the start. Log files contain commands and results tables, but not graphs.
To begin a log file, click the scroll-shaped Begin Log icon.
. and specify a name and folder
for the resulting log file. Alternatively, a log file could be started by choosing File - Log Begin from the top menu bar, or by typing a direct command such as
.

log using mondayl

1

Stata and Stata Resources

Multiple ways of doing such things are common in Stata. Each has its own advantages, and
each suits different situations or user tastes.
Log fi les can be created either in a special Stata format (.smcl), or in ordinary text or ASCII
fonnat (Jog). A .smcl (“Stata markup and control language”) file will be nicely formatted for
viewing or printing within Stata. It could also contain hyperlinks that help to understand
commands or error messages. Jog (text) files lack such formatting, but are simplerto use ifyou
plan later to insert or edit the output in a word processor. After selecting which type of log file
you want, click Save . For this session, we will create a .smcl log file named monday 1.smcl.
An existing Stata-format dataset named lofoten.dta will be analyzed here. To open or
retrieve this dataset, we again have several options:

i

select File - Open - lofoten.dta using the top menu bar;
| - lofoten.dta ; or

select

type the command use Lofoten.

Under its default Windows configuration, Stata looks for data files in folder C:\data. If the file
we want is in a different folder, we could specify its location in the use command,
.

use c:\books\sws8\chapter01\lofoten

or change the session s default folder by issuing a cd (change directory) command:
.

cd c:\books\sws8\chapter01\

.

use lofoten

Often, the simplest way to retrieve a file will be to choose File - Open and browse through
folders in the usual way.
To see a brief description of the dataset now in memory, type
.

describe

Contains data from C:\data\lofoten.dta
obs:
10
Jentoft & Kristoffersen '89
vars:
5
30 Jun 2005 10:36
s^ze:
130
(99.9% of memoryfree)
variable name
year
boats
men
penalty
decade

Sorted by:

I

£

3

storage
type

int
int
int
int
byte
decade

display
format

* 9. Og
*9.0g
•9.0g
•9.0g
•9.0g

value
label

variable label

decade

Year
Number of fishing boats
Number of fishermen
Number of penalties
Early 1970s or early 1980s

yea r

Many Stata commands can be abbreviated to their first few letters. For example, we could
shorten describe to just the letter d. Using menus, the same table could be obtained by
choosing Data - Describe data - Describe variables in memory - OK.
This dataset has only 10 observations and 5 variables, so we can easily list its contents by
typing the command list (or the letter 1 ; or Data - Describe data - List data - OK):

4

Statistics with Stata

list
year

boats

men

penalty

i

1971
1972
1973
1974
1975

1809
2017
2068
1693
144 1

5281
6304
6794
5227
4077

71
152
183
39
36

6. I
7. I
8. I
9. I
10 . I

1981
1982
1 983
1984
1985

1540
1689
1842
1847
1365

4033
4267
4 4 30
4 622
3514

11
15
34
74
15

1.
2
3.
4.
5.

I
I

decade |
------ I
1970s |
1970s |
1970s |
1970s |
1970s |
------ I
1980s |
1980s |
1980s |
1980s |
1980s |

Analysis could begin with a table of means, standard deviations, minimum values and
maximum values (type summarize or su; orselectStatistics-Summaries, tables, & iests
Nummary statistics - Summary statistics - OK):
summarize
Variable |
year
boats
men
penalty
decade

|
|
I
|
|

Obs

Mean

S t d. Dev .

Min

Max

10
10
10
10
10

1978
1731.1
4854.9
63
.5

5.477226
232.1328
1045.577
59.59493
.5270463

1971
1365
3514
11
0

1985
2068
6794
183
1
i

To print results from the session so ffar, bring the Results window to the front by clicking
on this window or on S| (Bring Results
i (Print).
To copy a table commands, or other information from the Results window into a word

on

W'nd0W *S ‘n fr°nt by clicklng on this window or

DrTth

s . Drag the mouse to select the results you want, right-click the mouse, and then choose
Copy Text from the mouse’s menu. Finally, switch to your word processor and at the desired
menubanrP0,nt'
PaSte °T C'ick 3 “cliPboard” “on on the word processor’s

that there were more penalties in the 1970s:
.

tabulate decade,

Early 19~0s |
or early |
19=0s |

sum(penalty)

Summary of Number of penalties
Mean
std. Dev.
Freq.

1970s |
1930s |

96.2
29.8

67.41439
26.281172

5
5

Total |

63

59.594929

10

_T
sam® table couId be obtained through menus: Statistics - Summaries, tables & tests
npnXS °ne/tw°;way ‘able of summary statistics, then fill in decade as variable 1, and
penalty as the variable to be summarized. Although menu choices are often straightforward to

Stata and Stata Resources

5

use, you can see that they tend to be more complicated to describe than the simple text
akX?
i°m
P01nu°n’ WC WiU f°CUS Primarily On the “mmands, mentioning menu
to a
r
°CCaS10nally- Fully exploring the menus, and working out how to usl them
mannaTrll
’Wil'be
t0 the reader- For similar reasons>
Stata reference
manuals likewise take a command-based approach.
Perhaps the number of penalties declined because fewer people were fishing in the 1980s
number
lumber of penalties correlates strongly ( r > .8) with the number of boats aid fishermen:'
correlate boats men penalty
(obs=10)

boats |
men |
penalty |

boats

men

penalty

1.0000
0.8748
0.8259

1.0000
0.9312

1.0000

A graph might help clarify these interrelationships. Figure 1.1 plots men and oenaltv

pro<,uce<lby,he g„ph twoway eonn.otfd

hand v axis °Ja
(l^'va'*‘ab,e>c011nec‘ed-Iine plot ofmen against year, using the lefthandy axis, yaxis (1). After the separator | | , we next ask for a connected-line plot of
‘his time using the right-handy axis, yaxis (2) . The resulting graph
visualizes the correspondence between the number of fishermen and the number of penalties
over time.
. graph twoway connected men year, yaxis(1)
I I

connected penalty year, yaxis(2)

o
o

o

Figure 1.1

ej

o

5S
i

5

s

fOSg

o Q.

5 o

0
X)

I

U)

.5?

0

h

E

o

sz

o

§

1970

o
1975
Year...
Number of fishermen

1980

1985

Number of penalties

to
eCaUFSetheyearS 197610 1980 aremissingin these data, Figure 1.1 shows 1975 connected
o l 98 . For some purposes, we might hesitate to do this. Instead, we could either find the
commands
UnCOnnected
issuing a slightly more complicated set of

6

Statistics with Stata

To print this graph, click on the Graph window or on gg| (Bring Graph Window to Front),
and then click the Print icon g

To copy the graph directly into a word processor or other document, bring the Graph
window to the front, right-click on the graph, and select Copy. Switch to your word processor
go to the desired insertion point, and issue an appropriate “paste” command such as Edit “ 'L PaJS‘2 SpeCia'(Metafile)’ or click a “clipboard” icon (different word processors
will handle this differently).

V

_ To save the graph for future use, either right-click and Save, or select File - Save Graph
from the top menu bar The Save As Type submenu offers several different fl)e formats t0
chose from. On a Windows system, the choices include
Stata graph (*.gph)

(A “live” graph, containing enough information for Stata to edit.)

As-is graph (*.gph)

(A more compact Stata graph format.)
Windows Metafile (*.wmf)
Enhanced Metafile (*.emf)

I

Portable Network Graphics (*.png)
TIFF(*.tif)
PostScript (*.ps)

I
r

'i

Encapsulated PostScript with TIFF preview (*.eps)
Encapsulated PostScript (*.eps)

I

Regardless of which graphics format we want, it might be worthwhile also to save a copy of our
graph in “live” .gph format. Live .gph graphs can later be retrieved, combined, recolored or
reformatted using the graph use or graph combine commands (Chapter 3).
ir

I
il i

Instead of using menus, graphs can be saved by adding a saving (filename) option
to any graph command. To save a graph with the filename figurel.gph, add another
separator | | ,acomma,and saving (figurel) . Chapter 3 explains more about the logic
of graph commands. The complete command now contains the following (typed in the Stata
Com mand window with as many spaces as you want, but no hard returns):
• graph twoway connected men year, yaxis(1)
II
connected penalty year, yaxis(2)
I I
, saving(figurel)

Through all of the preceding analyses, the log file mondayl.smcl has been storing our
results. There are several possible ways to review this file to see what we have done:
File - Log - View - OK
- View snapshot of log file - OK

1
II

L

typing the command view mondayl. smcl
We could print the log file by choosing g (Print) . Log files close automatically at the end
of a Stata session, or earl ier if instructed by one of the following:

Stata and Stata Resources

7

File - Log - Close

^>j|- Close log file - OK
typing the command log close

Once closed, the file mondayl.smcl could be opened again through File - View during a
subsequent Stata session. To make an output file that can be opened easily by your word
processor, either translate the log file from .smcl (a Stata format) to Jog (standard ASCII text
fonnat) by typing
translate mondayl.smcl mondayl.log

or start out by creating the file in Jog instead of .smcl format.

Stata’s Documentation and Help Files
The complete Stata 9 Documentation Set includes over 6,000 pages in 15 volumes: a slim
Getting Started manual (for example, Getting Started with Stata for Windows), the more
extensive User’s Guide, the encyclopedic three-volume Base Reference Manual, and separate
reference manuals on data management, graphics, longitudinal and panel data, matrix
programming (Mata), multivariate statistics, programming, survey data, survival analysis and
epidemiological tables, and time series analysis. Getting Started helps you do just that, with
the basics of installation, window management, data entry, printing, and so on. The User's
Guide contains an extended discussion of general topics, including resources and
troubleshooting. Of particular note for new users is the User's Guide section on “Commands
everyone should know.” The Base Reference Manual lists all Stata commands alphabetically.
Entries for each command include the full command syntax, descriptions of all available
options, examples, technical notes regarding formulas and rationale, and references for further
reading. Data management, graphics, panel data. etc. are covered in the general references, but
these complicated topics get more detailed treatment and examples in their own specialized
manuals. A Quick Reference and Index volume rounds out the whole collection.
When we are in the midst of a Stata session, it is often simpler to ask for onscreen help
instead of consulting the manuals. Selecting Help from the top menu bar invokes a drop-down
menu of further choices, including help on specific commands, general topics, online updates,
the Stata Journal, or connections to Stata’s web site (www.stata.com). Alternatively, we can
bring the Viewer ( ^!) to front and use its Search or Contents features to find information.
We can also use the help command. Typing help correlate , for example, causes
help information to appear in a Viewer window. Like the reference manuals, onscreen help
provides command syntax diagrams and complete lists of options. It also includes some
examples, although often less detailed and without the technical discussions found in the
manuals. The Viewer help has several advantages over the manuals, however. It can search
for keywords in the documentation or on Stata’s web site. Hypertext links take you directly to
related entries. Onscreen help can also include material about recent updates, or the
“unofficial” Stata programs that you have downloaded from Stata’s web site or from other
users.

8

Statistics with Stata

Searching for Info rm at io n
Selecting Help Search - Search documentation and FAQs provides a direct way to search for
information in State’s documentation or in the web site’s FAQs (frequently asked questions)
and other pages. The equivalent Stata command is
. search keywords

Options available with search allow us to limit our search to the documentation and FAQs,
to net resources including the Stata Journal, or to both. For example.
. search median regression

will search the documentation and FAQs for information indexed by both keywords, “median”
and regression. To search for these keywords across Stata’s Internet resources in addition
to the documentation and FAQs, type
. search median regression, all

Search results in the Viewer window contain clickable hyperlinks leadingto further information
or original citations.

One specialized use for the search command is to provide more information on those
occasions when our command does not succeed as planned, but instead results in one of Stata’s
cryptic numerical error messages. For example, typing the one-word command table
produces the error or “return code” r(100):
. table
varlist required
r(100);

The table <command....
evidently requires a list of variables. Often, however, the meaning of
an error message is less obvious. To learn more about what return code r( 100). refers
to,type
-----. search rc 100

Keyword search
Keywords:
Search:

rc 100
(1) Official help files. FAQs, Examples, SJs, and STBs

Search of official help files, FAQs, Examples, SJs, and STBs
[P]

error
Return code 10?
varlist required;
= exp required;
using required;
by() option required;
Certain commands require a varlist or another element of the
language.
The, message specifies the required item that was
missing from the
command
d you gave. See the command’s syntax
- ----diagram. F
’ example, merge requires using be specified; perhaps,
~or
you meant to 4.type append.Or, ranksum requires a by() option;
see [R] signrank.

(end of search)

Type help search for more about this command.

[
I

Stata and Stata Resources

Stata Corporation
For orders, licensing, and upgrade information, you can contact Stata Corporation bye-mail at
stata@stata.com
or visit their web site at
http.7/www.stata.com
Stata’s extensive web site contains a wealth of user-support information and links to resources.
Stata Press also has its own website, containing information about Stata publications includins
the datasets used for examples.


http://www.stata-press.com
Both web sites are well worth exploring.
The mailing or physical address is
Stata Corporation
4905 Lake way Drive
College Station, TX 7"845 USA
Telephone access includes an easy-to-remember 800 number.

telephone: 1-800-STATAPC
(1-800-782-8272)

fax:

U.S.

1-800-248-8272

Canada

1-979-696-4600

International

1-979-696-4601

Online updates within major versions are free to licensed Stata users. These provide a fast
and simple way to obtain the latest enhancements, bug fixes, etc. for your current version. To
find out whether updates> exist tor your Stata. and initiate the simple online update process
itself, type the command
update query

Technical support can Ibe obtained by sending e-mail messages with your Stata serial
number in the subject line to
tech_support@stata.com

Before calling or writing for technical help, though, you might want to look at
wwv.stata.com to see whether your question is a FAQ. The site also provides product
ordering, and help infonnation; international notes; and assorted news and announcements.
Much attention is given to user support, including the following:
FAQs — Frequently asked questions and their answers. If you are puzzled by something and
can t find the answer in the manuals, check here next — it might be a FAQ. Example questions
range from basic — “How can I convert other packages’ files to Stata format data files?" to
more technical queries such as "How do I impose the restriction that rho is zero using the
heckman command with full ml?”

1

9

UPDATES — Frequent minor updates or bug fixes, downloadable at no cost by licensed Stata
users.

10

Statistics with Stata

OTHER RESOURCES — Links and information including online Stata instruction
(NetCourses); enhancements from ths Stata Journal- an independent listserver (Statalist) for
discussions among Stata users; a bookstore selling books about Stata and other up-to-date
statistical references; downloadable datasets and programs for Stata-related books; and links
to statistical web sites including Stata’s competitors.
The following sections describe some of the most important user-support resources.

Statalist
Statalist provides a valuable online forum for communication among active Stata users. It is
m ependent of Stata Corporation, although Stata programmers monitor it and often contribute
to the discussion. To subscribe to Statalist, send an e-mail message to
maj ordomo@hsphsun2.harvard.edu

The body of this message should contain only the following words:
subscribe statalist

The list processor will acknowledge your message and send instructions for using the list,
including how to post messages of your own. Any message sent to the following address goes
out to all current subscribers:
statalist@hsphsun2.harvard.edu
Tk • D|0 nOt try t0 subscnbe or unsubscribe by sending messages directly to the statalist address.
This does not work, and your mistake goes to hundreds of subscribers. To unsubscribe from
the list, write to the same majordomo address you used to subscribe:
majordomo@hsphsun2.harvard.edu

but send only the message
unsubscribe statalist

or send the equivalent message
signoff statalist

If you plan to be traveling or offline for a while, unsubscribing will keep your mailbox from
tilling up with Statalist messages. You can always re-subscribe.
Searchable Statalist archives are available at
http://www.stata.com/statalist/archive

The material on Statalist includes requests for programs, solutions, or advice, as well as
answers and general discussion. Along with the S/t?rtf Jcwr/za/(discussed below), Statalist plays
a major role in extending the capabilities both of Stata and of serious Stata users.

The Stata Journal
From 1991 through 2001, a bimonthly publication called the Stata Technical Bulletin (STB)
served 35 a means of distributing new commands and Stata updates, both user-written and
o icial. Accumulated STB articles were published in book form each year as Stata Technical
Bulletin Reprints, which can be ordered directly from Stata Corporation.

I

Stata and Stata Resources
i

11

thm hh k ®rowthuof *e Bernet, instant communication among users became possible
ou0h vehicles such as Statalist. Program files could easily be downloaded fror/distant
sources. A bimonthly printed journal and disk no longer provided the best avenues either for
communicating among users, or for distributing updates and user-written programs. To adapt
to a changing world, the STB had to evolve into something new.
P
The Stata Journal was launched to meet this challenge and the needs of Stata’s broadening
users along with unofficial commands written by Stata Corporation employees New
commands are not its primary focus, however. The Stata Journal also contains refereed
expository artic es about statistics, book reviews, and a number of interesting columns
lanauaTf Th63?1"8 T3'3
NlchoIaS L Cox’ on effective use of the Stata programming
'
f J
u
‘S ‘ntended f°r nOvice as weI1 as experienced Stata users For
example, here are the contents from one recent issue:
“Exploratory analysis of single nucleotide polymorphism (SNP) for
M.A. Cleves
quantitative traits”
“Value label utilities: labeldup and labelrename”
J. Weesie
“Multilingual datasets”
J. Weesie
“Multiple imputation of missing values: update”
P.
Royston
^Estimation and testing of fixed-effect panel-data systems”
J.L.
Blackwell,
III
“Data inspection using biplots”
U. Kohler & M. Luniak
“Stata in space: Econometric analysis of spatially explicit raster data”

D. Muller
Using the file command to produce formatted output for other applications

“Mns smlsfa

ph)^,ans using

--------------- • Eplaymaker

“Speaking Stata: Density probability plots”


S.Lemesh«»&KL.Moeiehbi“
Linear, Logistic, Survival, and Repeated Measures Models"

The Stata Journal is published quarterly. Subscriptions
can be purchased directly from
Stata Corporation by visiting www.stata.com.

Books Using Stata
Statato 11X1
S.07£fe[encc manuals, a growing library of books describe Stata, oruse
aonhcat on
t.
Y ?techniqueS- These books include general introductions; disciplinary
applications such as social science, biostatistics or econometrics; and focused texts concerning
survey analysis, experimental data, categorical dependent variables, and other subjects Thf
Bookstore pages on Stata’s web site have up-to-date lists, with descriptions of content:'
http://www.stata.com/bookstore/

XSXZS ’ “nM' ,’i,cc ,<’ '“rn ato"1 “d

L

%
1

■■'"

•■''

v:
'
'
'

Data Management

S=S===EE;=ES~=

anoil'™/1!3 t'fan,sfer,program-translate the dataset directly from a system file created by
another spreadsheet database, or statical program. Once Stata has the data in memory we
can save the data in Stata format for easy retrieval and updating in the future.
Data management encompasses the initial tasks of creating a dataset, editing to correct
errors, and adding internal documentation such as variable and value labell It also
encompasses many other jobs required by ongoing projects, such as adding new observations
'ailables reorganmng, simplifying, or sampling from the data; separating, combing, or
lapsing datasets; converting variable types; and creating new variables through algebraic or
ogica! expressions. When data-management tasks become complex or repetitive, Stata users
analXl
bT" Pr°SramS t0
the
A,th°Ugh Sta,a is best
kr its
analytical capabiht.es, it possesses a broad range of data-management features as well This
chapter introduces some of the basics.
The
s Guide provides an overview of the different methods for inputting data
followed by eight roles for determining which input method to use. Input, editing, and many
7matI°;S d‘SC“SSed *" thls chaPter can be accomplished through the Data menus. Data
menu subheadings refer to the general category of task:
Describe data
Data editor

Data browser (read-only editor)
Create or change variables
Sort

Combine datasets
Labels
Notes

Variable utilities
Matrices

Other utilities

1

Data Management

13

Example Commands

I

append using olddata

Reads previously-saved dataset olddata.dta and adds all its observations to the data
currently in memory. Subsequently typing save newdata, replace will save the
combined dataset as newdata.dta.

I

I

browse

Opens the spreadsheet-like Data Browser for viewing the data. The Browser looks similar
to the Data Editor, but it has no editing capability, so there is no risk of inadvertently
changing your data. Alternatively, click ^|.

I

. browse boats men if year > 1980

Opens the Data Browser showing only the variables boats and men for observations in
which year is greater than 1980. This example illustrates the if qualifier, which can be
used to focus the operation of many Stata commands.
. compress

Automatically converts all variables to their most efficient storage types to conserve
memory and disk space. Subsequently typing the command save filename,
replace will make these changes permanent.
. drawnorm zl z2 z3, n(5000)

Creates an artificial dataset with 5,000 observations and three random variables, ~ 1. z2. and
z3, sampled from uncorrelated standard normal distributions. Options could specify other
means, standard deviations, and correlation or covariance matrices.
. edit

Opens the spreadsheet-like Data Editor where data can be entered or edited. Alternatively,
choose Window - Data Editor or click
edit boats year men

Opens the Data Editor with only the variables boats,year, and men (in that order) \ isibie
and available for editing.
. encode stringvar, gen(numvar)

Creates a new variable named numvar, with labeled numerical values based on the strins
(non-numeric) variable string\'ar.
.

format rainfall %8.2f

Establishes a fixed ( f) display format for numeric variable rainfall'. 8 columns wide, with
two digits always shown after the decimal.
. generate newvar = (x + y)/100

Creates a new variable named newvar, equal tox plusy divided by 100.
. generate newvar = uniform()

Creates a new variable with values sampled from a uniform random distribution over the
interval ranging from 0 to nearly 1, written [0,1).


infile x y z using data.raw

Reads an ASCII file named data.raw containing data on three variables: .v, y, and z. The
values of these variables are separated by one or more white-space characters — blanks,
tabs; and newlines (carriage return, linefeed, or both)
or by commas. With white-space

1 _

14

Statistics with Stata

delimiters, missing values are represented by periods, not blanks. With comma-delimited
data, missing values are represented by a period or by two consecutive commas. Stata also
provides for extended missing values, which we will discuss later. Other commands are
better suited for reading tab-delimited, comma-delimited, or fixed-column raw data; type
help infiling for more infomation.
list

Lists the data in default or “table” format. If the dataset contains many variables, table
format becomes hard to read, and list, display produces better results. See help
list for other options controlling the format of data lists.
.

list x y z in 5/20

Lists the x, y, and z values of the 5th through 20th observations, as the data are presently
sorted. The in qualifier works in similar fashion with most other Stata commands as
well.
. merge id using olddata

Reads the previously-saved dataset olddata.dta and matches observations from olddata
with observations in memory that have identical lvalues. Both olddata (the “using” data)
and the data currently in memory (the “master” data) must already be sorted by id.
.

replace oldvar = 100 * oldvar

Replaces the values of oldvar with 100 times their previous values.
sample 10

Drops all the observations in memory except for a 10% random sample. Instead of
selecting a certain percentage, we could select a certain number of cases. For example,
sample 55, count would drop all but a random sample of size n = 55.
.

save newfile

Saves the data currently in memory, as a file named newfile.dta. If newfile.dta already
exists, and you want to write over the previous version, type save newfile,
replace. Alternatively, use the menus: File - Save or File - Save As . To save
newfile.dta in the format of Stata version 7, type saveold newfile .
• set memory 24m

(Windows or Unix systems only) Allocates 24 megabytes of memory for Stata data. The
amount set could be greater or less than the current allocation. Virtual memory (disk space)
is used if the request exceeds physical memory. Type clear to drop the current data
from memory before using set memory.
.

sort x

Sorts the data from lowest to highest values of x. Observations with missing x values
appear last after sorting because Stata views missing values as very high numbers. Type
help gsort for a more general sorting command that can arrange values in either
ascending or descending order and can optionally place the missing values first.
.

tabulate x if y > 65

Pioduces a frequency table forx using only those observations that have^y values above 65.
The if qualifier works similarly with most other Stata commands.

I

I

I


Data Management

i

I

15

. use oldfile
Retrieves previously-saved Stata-format dataset oldfile.dta from disk, and places it in
memory. If other data are currently in memory, and you want to discard those data without
saving them, type use oldfile, clear . Alternatively, these tasks can be
accomplished through File - Open or by clickins G? •

Creating a New Dataset

I

dntaon Canadian provinces and territories listed in Table 2.1. (From the Federal, Provincial
x"
rr,t°rial AdvlSOry Committee on Population Health, 1996. Canada’s newest territory,
Nunavut, is not listed here because it
---------------- .t was part of the Northwest Territories until 1999 )
Table 2.1: Data on Canada and Its Provinces

1995 Pop.
Unemployment
(1000’s) _____ Rate (percent)

Place

Canada
Newfoundland
Prince Edward Island
Nova Scotia
New Brunswick
Quebec
Ontario
Manitoba
Saskatchewan
Alberta
British Columbia
Yukon
Northwest Territories

29606.1

Male Life
Expectancy
75.1
73.9
74.8
74.2
74.8
74.5
75.5
75.0
75.2
75.5
75.8
71.3
70.2

10.6
19.6

575.4
136.1

19.1

937.8
760.1
7334.2
11100.3

13.9
13.8
13.2
9.3

1137.5
1015.6
2747.0
3766.0

8.5
7.0
8.4
9.8

30.1

65.8

Female Life
Expectancy
81.1
79.8
81.3
80.4
80.6
81.2
81.1
80.8
81.8
81.4
81.4
80.4
78.0

The simplest way to create a dataset from Table 2.1 is through Stata’s spreadsheet-like Data
Editor, which is invoked either by clicking El, selecting Window - Data Editor from the top
menu bar or by typing the command edit. Then begin typing values for each variable in
tCanTdn *at Jtata aut°matlcally cal*s
var2, etc. Thus, varl contains place names
(Canada, Newfoundland, etc.); var2, populations; and so forth.

I

11

State Editov
Preserve!

< | » |
varltSj =

1^_

I

;

-

_T

1 ________ Canada

2

_____
--

Newfoundland

var2
29606.1 |
575.4 j

Hide |

Delete... |

var3
10.6.

vai»4
75.1
73.9

fea.;.v2<i
uarS
81.1
79.8

16

Statistics with Stata

We can assign more descriptive variable names by double-clicking on the column headinas
(such as varJ) and then typing a new name in the resulting dialog box — eight characters or
ewer works best, although names with up to 32 characters are allowed. We can also create
variable labels that contain a brief description. For example, var2 (population) might be
renamed pop, and given the variable label “Population in 1000s, 1995”.
Renaming and labeling variables can also be done outside of the Data Editor through the
rename and label variable commands:
.

I

rename var2 pop

. label variable pop "Population in 1000s, 1995"

Cells left empty, such as employment rates for the Yukon and Northwest Territories, will
automatically be assigned Stata’s system (default) missing value code, a period. At any time,
we can close the Data Editor and then save the dataset to disk. CIicking RTfl or W indow - D ata
Editor brings the Editor back.
If the first value entered for a variable is a number, as with population, unemployment, and
life expectancy, then Stata assumes that this column is a “numerical variable” and it will
thereafter permit only numerical values. Numerical values can also begin with a plus or minus
sign, include decimal points, or be expressed in scientific notation. For example, we could
represent Canada’s population as2.96061 e+7, which means 2.96061 x 107 or about 29.6 million
people. Numerical values should not include any commas, such as 29,606,100. If we did
happen to put commas within the first value typed in a column, Stata would interpret this as a
“string variable” (next paragraph) rather than as a number.
If the first value entered for a variable includes non-numerical characters, as did the place
names above (or “1,000” with the comma), then Stata thereafter considers this column to be a
string variable. String variable values can be almost any combination of letters, numbers,
symbols, or spaces up to 80 characters long in Intercooled or Small Stata, and up to 244
characters in Stata/SE. We can thus store names, quotations, or other descriptive information.
String variable values can be tabulated and counted, but do not allow the calculation of means,
correlations, or most other statistics. In the Data Editor or Data Browser, string variable values
appear in red, so we can visually distinguish the two variable types.
After typing in the information from Table 2.1 in this fashion, we close the Data Editor and
save our data, perhaps with the name canadaO.dta:
save canadaO

Stata automatically adds the extension .dta to any dataset name, unless we tell it to do
otherwise. If we already had saved and named an earlier version of this file, it is possible to
write over that with the newest version by typing
save, replace

At this point, our new dataset looks like this:

L

r

Data Management

describe
Contains data from C: \data\canadaO.dta
obs :
13
vars :
5
size:
(99.9* cf memory free)

variable name
var 1
Pop
va r 3
va r4
var5

storage
type
str21
float
float
float
float

display
format

value
label

3 Jul 2005 10:30

variable label

%21s
%9.0g
%9.0g
*9.0g
%9.0g

Population in 1000s,

1995

Sorted by:

list

1.

2.
3.
4.
5.
6.
7.
8.
9.
10 .

I

11 .
12 .
13 .

I
var 1
I -I
Canada
I
Newfoundland
I
Prince Edward Island
I
Nova Scotia
I
New Brunswick
I --I
Quebec
I
Ontario
I
Manitoba
I
Saskatchewan
I
Alberta
I --British Columbia
Yukon
I Northwest Territories

pop

var3

var 4

var5 |

-------- (
29606.1
575.4
136.1
937.8
760.1

10.6
19.6
19.1
13.9
13.8

75.1
73.9
74 . 8
74.2
74.8

81.1 i
79.8 I
81.3 |
80.4 |
80.6 |

7334.2
11100.3
1137.5
1015.6
2747

13.2
9.3
8.5
7
8.4

74.5
75.5
75
75.2
75.5

3766
30.1
65.8

9.8

81.2 I
81 . 1 I
80.8 I
81.8 I
81.4 I
—I
81.4 I
fin
o
u • * I
78 I

’’S . 8

71.3
■70.2

summarize

Variable

|

varl
pop
var3
var 4
var5



Jos

Mear.

Std. Dev.

Min

Max

13
11
13
13

4 5 54 . . 09
12.lOyoy
74.29231
80.71539

6 4.1 4.3 C 4
4.250048
1.673052
.975402-

30.1
7
70.2
78

29606.1
19.6
75.8
81.8

Examining
Examining such
such output
output tables
tables gi
gives us a chance to look for errors that should be corrected
> jsu™narlze table, for instance, provides several numbers useful in proofreading

“n (ca“da) ,h,t ~
The next step is to make our dataset more self-documenting. The variables could be given
more descriptive names, such as the following:
'
.

i

rename varl place
rename var3

unemp

17

18

1

Statistics with Stata

rename

var4 mlife

rename

var5 flife

Stata also permits us 1to add several kinds of labels to the data, label data describes
the dataset as a whole. For example,
.

label data "Canadian dataset 0"

label variable describes an individual variable. For example,
.

label variable place "Place name "

.

label variable

.

label variable mlife "Male life

I'

.

unemp "%

15+ population unemployed,

1995"

expectancy years"
label variable flife "Female life
expectancy years"

By labeling data and variables, we obtain a dataset that iis more self-explanatory:
.

describe

Contains data from C:\data\canada0 . dta
13
obs:
vars:
5
size:
533 (99.9% of
memory free)
variable name

storage
type

display
format

place
Pop
unemp

str21
float
float

%21s
%9.0g
%9.0g

mlife
flife

float
float

%9.0g
%9.0g

value
label

Canadian z taser

3 Jul 2C:= 10:45

variable label
Place n a r e
Populatio- in lOZZs, 1995
15+ pop;_ation unemployed,
1995
Male life expects’zy years
Female life expecz = r.cy years

Sorted by:

Once labeling is completed, we should save the data to disk by using File - Save or typing
save, replace

We can later retrieve these data any time through

File - Open, or by typing

. use c:\data\canadaO
(Canadian dataset 0)

e can then proceed with a new analysis. We might notice, for instance, that male and female
life expectancies correlate positively with each other and also negatively with the
unemployment rate. The life expectancy-unemployment rate correlation Is sliahtly stronger
for males.
. correlate unemp mlife flife
(obs=ll)

unemp|
mlife|
flife|

unemp

mlife

flife

1.0000
-0.7440
-0.6173

1.0000
0.7631

1.0000

r

Data Management

19

The order of observations within a dataset can be changed through the sort command.
For example, to rearrange observations from smallest to largest in population, type

I

.

sort pop

String variables are sorted alphabetically instead of numerically. Typing sort place will
rearrange observations putting Alberta first, British Columbia second, and so on.
We can control the order of variables in the data, using the order command, For
example, we could make unemployment rate the second variable, and population last:
order place unemp mlife flife pop

The Data Editor also has buttons that perform these functions. The Sort button applies to
the column currently highlighted by the cursor. The « and » buttons move the current
variable to the beginning or end of the variable list, respectively. As with any other editing,
these changes only become permanent if we subsequently save our data.
The Data Editor’s Hide button does not rearrange the data, but rather makes a column
temporarily invisible on the spreadsheet. This feature is convenient if, for example, we need
to type in more variables and want to keep the province names or some other case identification
column in view, adjacent to the “active” column where we are enterina data.
We can also restrict the Data Editor beforehand to work only with cenain variables, in a
specified order, or with a specified range of values. For example,
edit place mlife flife
or

edit place unemp if pop > 100

The last example employs an if qualifier, an important tool described in the next section.

Specifying Subsets of the Data:

in and if Qualifiers

Many Stata commands can be restricted to a subset of the data by adding an in or if
qualifier. (Qualifiers are also available for many menu selections: look for an if/in or by/if/in
tab along the top ot the menu.) in specifies the observation numbers to which the command
applies. Forexample, list in 5 tells Stata to list only the 5th observation. Tolistthelst
through 20th observations, type
list in 1/20

The letter 1 denotes the last case, and -4 , for example, the fourth-from-last. Thus, we could
list the four most populous Canadian places (which will include Canada itself) as follows:
.

sort pop

.

list place pop in -4/1

Note the important, although typographically subtle, distinction between 1 (number one, or
first observation) and 1 (letter “el,” or last observation).-The in qualifier works in a similar
way with most other analytical or data-editing commands! It always refers to the data as
presently sorted.

20

T

Statistics with Stata

The if qualifier also has broad applications, but it selects observations based on specific
variable values. As noted, the observations in canadaO.dta include not only 12 Canadian
provinces or territories, but also Canada as a whole. For many purposes, we might want to
exclude Canada from analyses involving the 12 territories and provinces. One way to do so is
to restrict the analysis to only those places with populations below 20 million (20,000
thousand); that is, every place except Canada:
summarize if pop < 20000

Variable I

Obs

Mean

place

|

Pop
unemp
mlife
flife

I
|
|
|

0
12
10
12
12

2467.158
12.26
74.225
80.6=333

Std.

Min

Max

3435.521
4 . 44877
1 . 728965
1.0116

30.1
7
70.2
78

19.6
75.8
81 . 8

Compare this with the earlier summarize output to see how much has changed. The
previous mean of population, for example, was grossly misleading because it counted every
person twice.

<
==

(is less than) sign is one of six relational operators:
is equal to

!=

is not equal to (

>

is greater than

<

is less than

>=

is greater than orequal to

<=

is less than or equal to

The

also works)

A double equals sign, “ == ”, denotes the logical test, ^Is the value on the left side the same
as the value on the right?” To Stata, a single equals sign means something different: "Make
the value on the left side be the same as the value on the right.” The single equals sign is not
a relational operator and cannot be used within if qualifiers. Single equals signs have other
meanings. They are used with commands that generate new variables, or replace the values of
old ones, according to algebraic expressions. Single equals signs also appear in certain
specialized applications such as weighting and hypothesis tests.

Any of these relational operators can be used to select observations based on their values
for numerical variables. Only two operators, = and ! =, make sense with string variables.
To use string variables in ani if qualifier, enclose the target value in double quotes. For
example, we could get a summary excluding Canada (leaving in the 12 provinces and
territories):
.

summarize if place

!=

"Canada"

Two or more relational operators can be combined within a single if expression by the
use of logical operators. State’s logical operators are the following:
&

and

I

or (symbol is a vertical bar, not the number one or letter “el”)
not
also works)

i

r

Data Management

21

The Canadian territories (Yukon and Northwest) both have fewer than 100,000 people. To find
the mean unemployment and life expectancies for the 10 Canadian provinces only, excluding
both the smaller places (territories) and the largest (Canada), we could use this command:
summarize unemp mlife flife if pop > 100

.

& pop < 20000

Variable |

Obs

Mean

Std. Dev.

Min

Max

unemp |
mlife |
flife |

10
10
10

12.26
74 . 92
80.98

4.44877
.6051633
.586515

7
73.9
79.8

19.6
75.8
81.8

I

Parentheses allow us to specify the precedence among multiple operators. For example,
we might list all the places that either have unemployment below 9, or have life expectancies
of at least 75.4 for men and 81.4 for women:
.

I

list if

8.
9.
10 .
11 .

I
I
I
I
I

unemp <

9

|

(mlife >= 75.4

& flife >= 81.4)

p ace

pop

unemp

mlife

Manitoba
Saskatchewan
Alberta

1137.5
1015.6
2747
3766

8.5
7
8.4
9.8

75
75.2
75.5
75.8

I British Colu-bia

flife |
----- I
80.8 |
81.8 |
•81.4 |
81.4 |

A note of caution regarding missing values: Stata ordinarily shows missing values as a
period, but in some operations (notably sort and if, although not in statistical calculations
such as means or correlations), these same missing values are treated as if they were large
positive numbers. Watch what happens ifwe sort places from lowest to highest unemployment
rate, and then ask to see places with unemployment rates above 15%:
sort unemp

.

list if unemp > 15

10 .
11.
12 .
13 .

I
place
I
Prince Edward Island
I
I
Newfoundland
I
Yukon
I Northwest Territories

Pop

unemp

mlife

136.1
575.4
30.1
65.8

19.1
19.6

74.8
73.9
71.3
70.2

----------- +
flife |
----------- I
81.3 |
79.8 |
80.4 |
78 |

The two places with missing unemployment rates were included among those “greater than 15.”
In this instance the result is obvious, but with a larger dataset we might not notice. Suppose
that we were analyzing a political opinion poll. A command such as the following would
tabulate the variable vote not only for people with ages older than 65, as intended, but also for
any people whose age values were missing:
.

tabulate vote if age > 65

Where missing values exist, we might have to deal with them explicitly as part of the if
expression.
.

tabulate vote if age > 65

& age <

.

A less-than inequality such as age < . is a general way to select observations with
nonmissing values. Stata permits up to 27 different missing values codes, although we are

22

Statistics with Stata

using only the default
. here. The other 26 codes are represented internally as numbers
even larger than “ . “.so < . avoids them all. Type help missing for more details.

The in and if qualifiers set observations aside temporarily so that a particular
command does not apply to them. These qualifiers have no effect on the data in memory, and
the next command will apply to all obsen ations, unless it too has an in or if qualifier. To
di op variables from the data in memory, use the drop command. For example, to drop mlife
and flife from memory, type
drop mlife flife

.

VV e can drop obsen ations from memory by using either the in qualifier or the if
qualifier. Because we <earlier sorted on unemp, the two territories occupy the 12th and 13th
positions in the data. Canada itself is 6th... One way to drop these three nonprovinces employs
the in qualifier, drop in 12/13 means ‘‘drop the 12th through the 13th observations.”
list

.

1
2
3

I
I
I
I

z

5
g
7
8
9
10

I
I
I
I
I
I

11
12
13

place

pcc

unemp |
---------- I

Saskatchewan
Alberta
Manitoba
Ontario
British 2oiumbia

1015.6
274"
1137.5
11100.3
3766

Canada
Quebec
New Brunswick
Nova Scotia
Prince Edward Island

19606.1
7334.2
760.1
937.S
136.1

Newftundland
Yukon
Northwest Territories

575.4
30.1
65.6

7 I
8.4 |
8.5 |
9.3 |
9.8 |
-----------I
10.6 |
13.2 |
13.8 |
13.9 |
19.1 |
---------- I
19.6 |
. I
. I

. drop in 12/13
(2 observations deleted)

. drop in 6
•1 observation deleted)

The
1 lie same
bdinc change
enange could
coum have
nave been accomplished through an if qualifier, with a command
that says “drop ifplace equals Canada or population is less than 100.”
drop if place == "Canada"
(3 observations deleted)

|

pop < 100

After dropping Canada, the territories, and the variables mlife and flife, we have the
following reduced dataset:
list
---------- +
I
I
1. I
2. I
3. I
4. I
5. I

place

POP

Saskatchewan
Alberta
Manitoba
Ontario

1015.6
2747
1137.5
11100.3
3766

British Columbia

unemp |
----------- I
7 I
8.4 |
8.5 |
9.3 |
9.8 |

L

Data Management

6
7

8
9
10

!

Quebec
i
i
I
I

New Brunswick
Nova Scotia
Prince Edward Island
Newfoundland

7334.2
760.1
937.8
136.1
575.4

13.2
13.8
13.9
19.1
19.6

23

I
I
I
I
I
I

We can also drop selected variables or observations through the Delete button in the Data
Editor.

snecifvwhich m k"8
Va?ab‘eS °r observations t0 droP> * sometimes is simpler to
specify which to keep. The same reduced dataset could have been obtained as follows:

I

.

keep place pop unemp

■ keep if place != "Canada"
”Canada"
(3 observations deleted)

&
& pop
pop >=
>= 100
100

Like any other changes to the data in memory, none of these reductions affect disk files
( save' TepilStan^to31 S°T’ Wil*
°PtiOn °f Writing °Ver the oId dataset
new Zp th ?
destroying it, or just saving the newly modified dataset with a
new name (by choosing File - Save As , or by typing a command with the form
save
newname) so that both versions exist on disk.

Generatin

and Replacing Variables

generate and replace commands allow us to create new variables or change the
va ues of existing variables. For example, in Canada, as in most industrial societies, women
n o live longer than men. Toanalyze regional variations in this gender gap we might
re neye dataset cgWg^and^^new variable equal to female life

(We). m the main P^fr^^r^^e
statement (unlike if qualifiers) we use a single equals sign.
- use canadal, clear
(Canadian dataset 1)

. generate gap = flife

.

mlife

label variable gap "Female-male
gap life expectancy"
describe

Contains data from C:\data\canadal.dta
obs:
13
vars:
6
size:
595
(99.9% of memory free)
variable name

storage
type

display
format

place
POP
unemp

str21
float
float

%21s
% 9.0g
%9.0g

mlife
flife

float
float
float

%9.0g
%9.0g
%9.0g

gap
Sorted by:

value
label

Canadian dataset 1
3 Jul 2005 10:48

variable label

Place name
Population in 1000s, 1995
% 15+ population unemployed,
1’995
Male life expectancy years
Female life expectancy years
Female-male gap life expectancy

I!

24

Statistics with Stata

!r


list place flife mlife gap

-------- piace

flife

mlife

Canada
-Indiana

81.1

Prince Edward Island
Nova Scotia
New Brunswick

81 . 3
80 . 4
80 . e

75.1
73.9
74 . 8
74.2
74 . 8

»cebec
Ontario
Manitoba
Sas ka tone war.
Alberta

81.2
81 .1
80 . =
81 . 5
81 . 4

74 . 5
75.5
75
75.2
75.5

81
80

75.8
71.3
70.2

I --

British

i

•Jir.bia
Yukon
Northwest Territories

gap |
--------- (
6 I
5.900002 |
6.5 |
6.200005 |
5.799995 |
--------- !
6.699997 |
5.599998 |
5. ■800003 |
6.600006 |
5.900002 |
I
5.599998 |
9.099998 |
7.800003 |

For the province of Newfoundland, the true value ofgap should be 79.8 -73 9 = 5 9 years
but the output shows this value as 5.900002 instead. Like all computer programs, Stata stores
numbers in binary form, and 5.9 has no exact binary representation. The small inaccuracies that
arise from approximating decimal fractions in binary are unlikely to affect statistical
calculations much because calculations are done in double precision (8 bytes pernumber).
The) appear disconcerting in data lists, however. We can change the display format so that
Stata shows only a rounded-off version. The following command specifies a fixed display
format tour numerals wide, with one digit to the right of the decimal:
.

format gap %4.1f

observations1116 dlSP'ay Sh°WS 5'9’ however>a command such as the following will return no
list if gap == 5.9

This occurs because Stata believes the value does not exactly equal 5.9. (More technically
Stata stores gap values in single precision but does all calculations iin double, and the singleand double-precision approximations of 5.9 are not identical.)
Display formats, as well as variables names and labels, can also be changed by double­
clicking on a column in the Data Editor. Fixed numeric formats such as %4.1f are one of
the three most common numeric display format types. These are
% iv. 5g

% w. 5f
% w. 5e

General numeric format, where iv specifies the total width or number ofcolumns
displayed and d the minimum number of digits that must follow the decimal
point. Exponential notation (such as 1.00e+07, meaning 1.00 x 107 or 10 million)
and shifts in the decimal-point position will be used automatically as needed, to
display values in an optimal (but varying) fashion.
Fixed numeric format, where w specifies the total width or number of columns
displayed and d the fixed number of digits that must follow the decimal point.
Exponential numeric format, where w specifes the total width or number of
columns displayed and d the fixed number of digits that must follow the decimal
point.

Data Management

25

For example, as we saw in Table 2.1, the 1995 population of Canada was approximately
29,606,100 people, and the Yukon Territory population was 30,100. Below we see how these
two numbers appear under several different display formats:
format
Canada
Yukon______
%9.0g
2.96e+07
30100
v
%9.1f
29606100.0
30100.0
%12.5e
2.9606 le+07
3.01000e+04
Although the displayed values look different, their internal values are
are identical.
identical. Statistical
Statistical
calculations remain unaffected by display formats. Other numerical display formatting options
include the use <of commas, left- and right-justification, or leading zeroes. There also exist
special formats for dates, time: series variables, and string variables. Type help formatfor more information.

replace can make the same sorts of calculations as generate, but it changes values
of an existing variable instead of creating a new variable. For example, the variablepop in our
dataset gives population in thousands. To convert this to simple population, w e just multiply
(“ * ” means multiply) all values by 1,000:
.

replace pop = pop * 1000

replace can make such wholesale changes, or it can be used with in or if qualifiers
to selectively edit the data. To illustrate, suppose that we had questionnaire data with variables
including age and year bom (born). A command such as the following would correct one or
more typos where a subject’s age had been incorrectly typed as 229 instead of 29:
.

replace age = 29 if age == 229

Alternatively, the following command could correct an error in the value ofage for observation
number 1453:
.

replace age = 29 in 1453

For a more complicated example,
.

replace age = 2005-born if age >= .

|

age < 2005-born

This replaces values ofvariable age with 2005 minus the year of birth ifage is missing or if the
reported age is less than 2005 minus the year of birth.
generate and replace provide tools to create categorical variables as well. We
noted earlier that our Canadian dataset includes several types of observations: 2 territories. 10
provinces, and 1 country combining them all. Although in and if qualifiers allow us to
separate these, and drop can eliminate observations from the data, it might be most
convenient to have a categorical variable that indicates the observation’s “type.” The fol lowing
example shows one way to create such a variable. We start by generating type as a constant
equal to 1 for each observation. Next, we replace this with the value 2 for the Yukon and
orthwest Territories, and with 3 for Canada. The final steps involve labeling new variable
type and defining labels for values 1, 2, and 3.
. use canadal, clear
(Canadian dataset 1)
. generate type = 1

J

26

Statistics with Stata

.

replace type = 2 if place
Territories"
(2 real changes made)

"Yukon”

. replace type = 3 if place
(1 real change made)

"Canada"

.

label variable type "Province,

.

label values type typelbl

| place

"Northwest

territory or nation"

label define typelbl 1 "Province" 2 "Territory" 3 "Nation"
list place flife mlife gap type

3

i
I
I
I
I
I
I

place

flife

mlife

gap

Canada
Newfoundland
Prince Edward Island
Nova Scotia
New Brunswick

81.1
79.8
81.3
80.4
80.6

75.1
73.9
74.8
74.2
74.8

6
5.900002
6.5
6.200005
5.799995

Quebec
Ontario
Manitoba
Saskatchewan
Alberta

81.2
81.1
80.8
81.8
81.4

74.5
75.5
75
75.2
75.5

6.699997
5.599998
5.800003
6.600006
5.900002

81.4
80.4
78

75.8
71.3
70.2

5.599998
9.099998
7.800003

6
3
9
10
11
12
13

I
i
I
British Columbia
I
Yukon
I Northwest Territories

type I
----------I
Nation |
Province I
Province I
Province |
Province I
--------- I
Province |
Province |
Province |
Province |
Province |
--------- I
Province I
Territory |
Territory |

As illustrated, labeling the values of a categorical variable requires two commands. The
label define command specifies what labels go with what numbers. The label
values command specifies to which variable these labels apply. One set of labels (created
through one label define command) can apply to any number of variables (that is, be
referenced in any number of label values commands). Value labels can have up to
32,000 characters, but work best for most purposes if they are not too long.

generate can create new variables, and replace can produce new values, using any
mixture of old variables, constants, random values, and expressions. For numeric variables, the
following arithmetic operators apply:
+ add
subtract


multiply

/

divide

A

raise to power

Parentheses will control the order of calculation. Without them, the ordinary rules of
precedence apply. Of the arithmetic operators, only addition, “4- ”, works with string variables,
where it connects two string values into one.
Although their purposes differ, generate and replace have similar syntax. Either
can use any mathematically or logically feasible combination of Stata operators and in or
if qualifiers. These commands can also employ Stata’s broad array of special functions,
introduced in the following section.

Data Management

27

Using Functions
This section lists many of the functions available for use with generate or replace
For example, we could create a new variable named loginc. equal to the natural logarithm of
mcome, by using the natural log function In within a generate command:
. generate loginc = In(income)

In is one of Stata’s mathematical functions. These functions are as follows:
abs(x)

Absolute value of.v.

acos(x)

asin(x)

Arc-cosine returning radians. Because 360 degrees =
radians,
acos (x) *180/_j>i gives the arc-cosine returning degrees ( _pi
denotes the mathematical constant k).
Arc-sine returning radians.

atan(x)

Arc-tangent returning radians.

atan2(y,x)

Two-argument arc-tangent returning radians.

atanh(x)

Arc-hyperbolic tangent returning radians.
Integer n such that n-1 <.v < n

ceil(x)

cloglog(x)

I
I

comb(n,k)

Complementary log-log of.v:
ln(—ln( 1 —a))
Combinatorial function (number of possible combinations of n things
taken k at a time).

cos(x)

Cosine of radians. To find the cosine of v degrees, type

digamma(x)

</lnr(.v) / dx

exp(x)

Exponential (e to power).

floor(x)

Integer n such that n < .v < //-I

trunc(x)

Integer obtained by truncating .v towards zero.
Inverse of the complementary log-log: I - exp(-exp(x))
Inverse of logit of.v:
exp(.v) (1 e.xp(.v))
Natural (base e) logarithm. For any other base number B, to find the
base B logarithm of.v, type

generate y = cos(y *_pi/180)

invcloglog(x)
invlogit(x)

In (x)
i

generate y = ln(x)/ln(B)

I

Infactorial(x)

J

I

generate y = round(exp(Infact(x),1)

Ingamma(x)

Natural log of r(x). To find F(x), type

log(x)

Natural logarithm; same as in (x)
'Base 10 logarithm.

generate y - exp(Ingamma(x))
loglO(x)
logit(x)
max(xl,x2,..,xn)
min(xl,x2,..,xn)

1

Natural log of factorial. To find .v factorial, type

Log of odds ratio ofx: In(x /(l^r))
Maximum ofx/,x2, ...,xn.
Minimum ofx/, x2, ..., xn

28

Statistics with Stata

mod(x,y)

Modulus of a- with respect toy.

reldif(x,y)

Relative difference: j x - v | / (| y | + 1)
Round x to nearest whole number.
Round x in units of r.

round(x)
round(x,y)

sign(x)
sin(x)
sqrt(x)

total(x)

tan(x)
tanh(x)
trigamma(x)

-1 ifx<0, 0 ifx=0, +1 ifx>0
Sine of radians.
Square root.

Running sum ofx (also see help egen )
Tangent of radians.
Hyperbolic tangent ofx.
d2 lnF(x) / dx2

Many probabilityfunctions exit as well, and are listed below. Consult help probfun
and the reference manuals for important details, including definitions, constraints on
parameters, and the treatment of missing values.
betaden(a,b,x)

Probability density of the beta distribution.

Binomial(n ,k, p)

Probability of k or more successes in n trials when the probability of
a success on a single trial is p.

binormal(h,k,r)

Joint cumulative distribution of bivariate normal with correlation r.

chi2 (n, x)

Cumulative chi-squared distribution with n degrees of freedom.

chi2tail(n,x)

Reverse cumulative (upper-tail, survival) chi-squared distribution with
n degrees of freedom..
chi2tail(/^v) = 1 - chi2(?ux)
Partial derivative of the cumulative gamma distribution gammap(tf.x)
with respect to a.

dgammapda(a,x)
dgammapdx(a,x)

Partial derivative of the cumulative gamma distribution gammap(aa')
with respect to x.

dgammapdada(a,x)

2nd partial derivative of the cumulative gamma distribution
gammap(a,x) with respect to a.

dgammapdadx (a,x)

2nd partial derivative of the cumulative gamma distribution
gammap^^r) with respect to a and x.

dgammapdxdx(a,x)

2nd partial derivative of the cumulative gamma distribution
gammap(fl,x) with respect to x.
Cumulative F distribution with nl numerator and n2 denominator
degrees of freedom.

Y(nl,n2,f)

Fden(nl,n2,f)
Ftail(nl,n2,f)

Probability density function for the F distribution with nl numerator
and n2 denominator degrees of freedom.
Reverse cumulative (upper-tail, survival) F distribution with nl
numerator and n2 denominator degrees of freedom.
Ftai\(nfn2f) = 1 - Y(nfn2j)

Data Management

29

gammaden (a,b, g, x) Probability density function for the gamma
gamma family,
family. where
gammaden(aJ.O.r) = ithe probability density function for
the
cumulative gamma distribution gammap(^rv).
gammap (a, x)
Cumulative gamma distribution fora: also known as the incomplete
gamma function.
ibeta(a,b,x)

Cumulative beta distribution fort?, b; also know:n as the incomplete
beta function.

invbinomial(n,k,P)i

I

I

Inverse binomial. For/>< 0.5. probability p such that the
probability of observing k or more successes in n trials is P: for P >
0.5, probability p such that the probability of observing k or fewer
successes in n trials is 1 - P.
invchi2(n,p)
Inverse of chi2(). If chi2(/?rr) =/?. then invchi2(n,p) = .v
invchi2tail (n,p)
Inverse of chi2taii()
If chi2tail(n^v) = p,
then invchi2tail(//,p) = .V
invF(nl,n2,p)
Inverse cumulative F distribution.
If T(nl,n2f)=p,
then invF(/7/./72.p) =/
invFtaii (ni,n2,p) Inverse reverse cumulative F distribution.
If Ftail^z/^j/) =p,
then invFtail(/7/,/72,p) =f
invgammap(a,p)
Inverse cumulative gamma distribution.
I f gammap^^x) = p,
then invgammap(a./7) = x
invibeta(a,b,p)
Inverse cumulative beta distribution.
If ibeta(a,Z>^) =p,
then invibeta(a,/>,p) = x
invnchi2(n,L,p)
Inverse cumulative noncentral chi-squared distribution.
If nchi2(/7,Z,rx) =p,
then invnchi2(n,L,p) = x
invnFtaii (nl fn2tL,P) Inverse reverse cumulative noncentral Fdistribution.
If nFtaiI(w/,w2X /) =p,
then in\TiFtail(w7,n2X,p) = f
invnibeta(a,b,L,p)

Inverse cumulative noncentral beta distribution.
If nibeta(a,Z>/.A) =p, then invnibeta(a,Z>/ ,p) = x
invnormal(p)
Inverse cumulative standard normal distribution.
If normalfz) = p,
then invnormal(p) = z
invttail(n,p)
Inverse reverse cumulative Student’s t distribution.
If ttail(/?,r) =p,
then invttail(/7,p) = t
nbetaden(a,b,L,x ) Noncentral beta density with shape parameters a, b. noncentrality
parameter/,.
nchi2(n,L,x)

Cumulative noncentral chi-squared distribution with n degrees of
freedom and noncentrality parameter L.
nFden(nl,n2,L,x)
Noncentral F density with nl numerator and n2 denominator degrees
of freedom, noncentrality parameter L.
nFtaii (nl ,n2,L,X) Reverse cumulative (upper-tail, survival) noncentral F distribution
with nJ numerator and n2 denominator degrees of freedom,
noncentrality parameter L.

j

30

T

Statistics with Stata

nibeta(a fb,L,x)

normal(z)
normalden(z)

Cumulative noncentral beta distribution with shape parameters ci and
b, and noncentrality parameter L.
Cumulative standard normal distribution.
Standard normal density, mean 0 and standard deviation 1.

normalden(z,s)

Normal density, mean 0 and standard deviation s.

normalden(x,m,s)

Normal density, mean m and standard delation 5.
Noncentrality parameter L for the noncentral cumulative chi-squared
distribution.
If nchi2(/?,Z,rr) = p,
then npnchi2(/7.v.p) = L
Probability density function of Student's t distribution with n degrees
of freedom.

npnchi2(n,x,p)

tden (n, t)

ttail(n,t)

uniform()

I

I

Reverse cumulative(upper-tail) Student’srdistribution with/? degrees
of freedom. This function returns probability T> t.
Pseudo-random number generator, returning values from a uniform
distribution theoretically ranging from 0 to nearly 1, written [0,1).

Nothing goes inside the parentheses with uniform (). Optionally, we can control the
K
? gener"tOr’S StartinS seed’ and ^nce the stream of “random” numbers, by first
issuing a set seed # command — where # could be any integer from 0 to 2 31 - 1
inclus^e Onuttmgthe set seed command corresponds to set seed 123456789
which will always produce the same stream of numbers.

ran H!a?PrjV'dX”°re,^“40rf“''/“"”“anddate',el“,ed™“
A listing
can be foundh„ Chapter27 of the
nr by typing help a.i.tep. Beiowar|

daX" 1196o“,i0"S' “ElaPSed

'h'“

—T

II

date (slzS2[ ,y]) Elapsed date corresponding to 5,. 5, is a string triable indicating the date
in virtually any format. Months can be spelled out, abbreviated to three
characters, or given as numbers; years can include or exclude the century
anks and punctuation are allowed, s, is anv permutation of m d. and
L##]y with their order defining the order that month, day and year occur in
S|. ## gives the century for two-digit years in s,: the default is 19y.
d(l)
A date literal convenience function. For example, typing d(2janl960) is
IS
equivalent to typing 1.
mdy(m,d,y)
Elapsed date corresponding to m. d. and y.
day(e)

Numeric day of the month corresponding to e, the elapsed date.

month(e)

Numeric month corresponding to e. the elapsed date.
Numeric year corresponding to e, the elapsed date.

year(e)

dow(e)
doy(e)

week(e)
quarter(e)
halfyear(e)

Numeric day of the week corresponding to e, the elapsed date.
Numeric day of the year corresponding to e, the elapsed date.
Numeric week of the year corresponding to e, the elapsed date.
Numeric quarter of the year corresponding to e, the elapsed date.
Numeric half of the corresponding to e, the elapsed date.
1,

Data Management

31

Some useful special functions include the following:

autocode (x, n, x~ i n, xmax) Forms categories from x by partitioning the interval from xmin
to xmax into n equal-length intervals and returning the upper bound of the
interval that contains x.
cond(x,a,b)
Returns a if x evaluates to “true” and b if x evaluates to “false.”
. generate y = cond(incl > inc2, incl, inc2)

creates the variabley as the maximuni of incl and inc2 (assuming neither
is missing).

Creates a categorical variable that divides the data as presently sorted into
x subsamples that are as nearly equal-sized as possible.
Returns the integer obtained by truncating (dropping fractional parts of)

group(x)
trunc(x)

max(x , x ,

., x.) Returns the maximum ofx,, x2,..., xn. Missing values are ignored.
For example, max(3+2,1) evaluates to 5.

min (x , x ,, x_) Returns the minimum ofx! ,x2,... ,xn.
recode (x, x:, x2, . . . , x.) Returns missing ifx is missing, x, ifx < Xj, orx2 ifx <x2, and
so on.
round(x,y)

Returns x rounded to the nearest y.

sign(x)

Returns - 1 ifx < 0, 0 ifx = 0, and+1 ifx > 0 (missing ifx is missing).

total(x)

Returns the running sum ofx, treating missing values as zero.

Stringfunctions, not described here, help to manipulate and evaluate string variables. Type
help str fun for a complete list of string functions. The reference manuals and User's
Guide give examples and details of these and other functions.
Multiple functions, operators, and qualifiers can be combined in one command as needed.
The functions and algebraic operators just described can also be used in another way that does
not create or change any dataset variables. The display command performs a single
calculation and shows the results onscreen. For example:
. display 2+3
c.

. display logl0(10A83)
83
. display invttail (120, .025) * 34.1/sqrt(975)
2.1622305

Thus, display works as an onscreen statistical calculator.

Unlike a calculator, display , generate , and replace have direct access to
Stata’s statistical results. For example, suppose that we summarized the unemployment rates
from dataset Canadal.dta:
.

summarize unemp
Variable I

Obs

Mean

Std. Dev.

Min

Max

unemp |

11

12.10909

4.250048

7

19.6

After summarize_^Stata temporarily stores the mean as a macro named r (mean) .

32

Statistics with Stata

display r(mean)
12.109091

We could use this result to create variable unempDEV, defined as deviations from the mean:
. gen unempDEV = unemp - r (mean)
(e missing values generated)

summ unemp unempDEV
Variable I

Obs

Mean

unemp |
unempDEV I

il

12.10909
4.33e-08

11

Dev.

Min

Max

4.250048
4.250048

7
-5.-109091

19.6
7.49091

Srd.

Stata also provides another variable-creation command,
egen (“extensions to
generate ”), which has its own set of functions to accomplish tasks not easily done by
generate . These include such things as creating new variables from the sums, maxima,
minima, medians, interquartile ranges, standardized values, or moving averages of existing
variables or expressions. For example, the following command creates'a new variable named
zscore, equal to the standardized (mean 0, variance 1) values ofx:
. egen zscore = std(x)

Or, the following command creates new variable avg, equal to the row mean of each
observation s values on .v, y\ z, and w, ignoring any missing values.
. egen avg = rowmean(x,y,z, w}

To create a new variable named sum, equal to the row sum of each observation’s values onx
v, z, and w, treating missing values as zeroes, type
. egen sum = rowsum(x,y,z, w)

The following command creates new variable xrank, holding ranks corresponding to values of
a. xrank-I for the observation with highest.r. xrank = 2 for the second highest, and so forth.
. egen xrank = rank(x)

Consult help egen for a complete list of egen functions, or the reference manuals for
further examples.

Converting between Numeric and String Formats
Dataset canada2.dta contains one string variable, place. It also has
a labeled categorical
xariable, type. Both seem to have nonnumerical values.
. use canada2, clear
(Canadian dataset 2)
• list place type
I
I
1. I
2. I
3. I
4. I

place
Canada
Newfoundland
Prince Edward Island
Nova Scotia

type |
-------- !
Nation |
Province |
Province |
Province |

r

Data Management
5.

'.'ex Brunswick

33

Province |

---------------- (
6.

Quebec
tar io
itoba
Saskatchewan
Alberta

6.
9.
10.

I
I
II
British Columbia
I
Yukon
I Northwest Territories

11 .
12 .
13 .

Province
Province
Province
Province
Province

-------- !

|
|
|
I
|

Province |
Territory |
Territory |

-------- -

Beneath the labels, however, type remains a numeric variable, as we can see if we ask for the
no label option:
.

list place type, nolabel
place
1.
2.
3.
4.
5.

I

6.
7.
8.
9.
10.

ii.
12 .
13 .

I

l

I
I

I
I
I

type

|
■ I

Canada
Newfoundland
Prince Edward Island
Nova Scotia
New Brunswick

Quebec
Ontario
Manitoba
Saskatchewan
Alberta

I
I
I
II
British Columbia
I
Yukon
I Northwest Territories

3 I
1 I
1 I
1 I
1 I
I
1 I
1 I
1 I
1 I
1 I
I
1 I
2 I
2 I

String and labeled numeric variables look similar when listed, but they behave differently
when analyzed. Most statistical operations and algebraic relations are not defined for string
™nableS) so we miSht want t0 have both string and labeled-numeric versions of the same
formation in our data. The encode command generates a labeled-numeric variable from
a stnng variable. The number 1 is given to the alphabetically first value of the string variable
to the second, and so on. In the following example, we create a labeled numeric variable
named placenum from the string variable place'.
.

encode place,

gen(placenum)

The opposite conversion is possible, too: The decode command generates a string

? 1,be“<l n"n,eric v"“b"-Here"

z
.

decode

type,

gen(typestr)

When listed, the new numeric variable placenum, and the new string variable typestr, look
similar to the originals:


1

34

Statistics with Stata

list place placenum type

typestr

place

4

type

typestr |
----------- ,

Panada
N'-. wi i i.-.dland
Prince Edward Island
Nova Scotia
N e w B r un s w i c k

Canada
Newfoundland
i.-.ce Eaward Island
Nova Scotia
New Brunswick

Nation
Province
Province
Province
Province

Nation I
Province |
Province I
Province |
Province I
----------- I

Quebec
1ntario
Mar. i t oba
Saskatchewan
Alberta

Quebec
Ontario
Manitoba'
Saskatchewan
Alberta

Province
Province
Province
Province
Province

rrtish Columbia
Y ukon
nwest Territories

Province
Territory
Territory

Province I
Province |
Province I
Province |
Province |
---------- I
Province |
Territory |
Territory |

10
11
12
13

place­

British

tlumbia
Y ukon
.’c rthwest Territories

But with the nolabel option, the differences become visible. Stata views placenum and
type basically as numbers.
.

list place placenum type

typestr.

nolabel

place

placenum

Panada
Newfoundland
Prince Edward Island
Nova Scotia
New Brunswick

.oo: zooooooooooe+oo
. oo:: oooooo ooo oe-f-oo
1 .OOCOOOOOOOOOOOe + Ol
5 .OOOOOOOOOOOOOOe + OO
: .000000000000006 + 00

1
1
1
1

6.
7.
8.
9.
10 .

Quebec
On t a r i o
Manitoba
Saskatchewan
Alberta

1 . lOOOOOOOOOOOOOe + Ol
9.000300000 00000e + 00
4.00000000000000e+00
1 .2000-0000000000e + 01
1.00000000000000e + 00

1
1
1
1
1

12 .

clumbia
Yukon
_t tries

2 -OC-COOCOCOOOOOOe + OO
1.30000000000000e + 01
" .000 * 00000000006 + 00

1
2
2

I --

2
3.
4.
5.

I
I--

thwest

type

typestr I
I
Nation I
Province I
Province I
Province I
Province I
I
Province I
Province I
Province I
Province I
Province I
I
Province I
Territory |
Territory |

Statistical analyses, such as finding means and standard deviations, work only with
basically numeric variables. For calculation purposes, numeric variables’ labels do not matter.
summarize place placenum

Variable

Obs

place I
placenum I
type I
typestr I

0
13
13
0

I

type typestr

Me ar.

Dev .

Min

Max

1.307692

3.-9444
. 63: 4252

1
1

13
3

Occasionally we encounter a string variable where the values are all or mostly numbers.
To convert these string values into their numerical counterparts, use the real function. For
example, the variable siblings below is a string variable, although it only has one value, “4 or
more,'’ that could not be represented just as easily by a number.

1

Data Management

35

. describe siblings

str9

1. siblings

%9s

Number of siblings (string)

. list

I
siblings
--- I
I—
1. I
0
2. I
1
3. I
2
4. I
3
5. I 4 or more

|

I
I
I
I
|

. generate sibnum = real(siblings)
(1 missing value generated)

The new variable sibnum is numeric, with a missing value where siblings had “4 or more.”
list

1.
2.
3.

4.
5.

I
s iblings
I—
l
0
I
1
2
l
3
I
I 4 or more

sibnum |
----------- I
0 I
1 I
2 I
3


I
I

The destring command provides a more flexible method for converting string
variables to numeric. In the example above, we could have accomplished the same thing by
typing
destring siblings, generate(sibnum)

force

See help destring for information about syntax and options.

Creating New Categorical and Ordinal Variables
A previous section illustrated how to construct a categorical variable called type to distinguish
among territories, provinces, and nation in our Canadian dataset. You can create categorical
or ordinal variables in many other ways. This section gives a few examples.
type has three categories:
tabulate type

Province,
|
territory or|
nation
|

Freq.

Percent

Cum.

Province |
Territory |
Nation |

10
2
1

7 6.92
15.38
7.69

76.92
92.31
100.00

Total |

13

100.00

For some purposes, we might want to re-express a imulticategory variable as a set of
dichotomies or ‘‘dummy variables/’ each coded 0 or 1.. tabulate will create dummy

36

Statistics with Stata

variables automatically if we add the generate option. In the following example, this
results in a set of variables called typel, type2, and type3, each representing one of the three
categories of type:
tabulate type, generate(type)
Province,
|
territory or|
nation
|
Freq.
Percent

Province I
Territory |
Nation |

10
2
1

76. 92
15.38
7.69

Total |

13

100.00

Cum.

76.92
92.31
100.00

describe

.

Contains data from C:\data\canada2.dta
obs :
13
vars :
10
size:
637 (99.9% of memory free)
variable name

storage
type

display
format

place
POP
unemp

str21
float
float

%21s
%9.0g
%9.0g

ml i f e
flife
gap
type
typel
type2
type3

float
float
float
byte
byte
byte
byte

%9.0g
%9.0g
%9.0g
%9.0g
%8.0g
%8.0g
%8.0g

Sorted by:
Note:
.

value
label

typelbl

Canadian dataset 2
3 Jul 2005 10:48

variable label
Place name
Population in 1000s, 1995
% 15+ population unemployed,
1995
Male life expectancy years
Female life expectancy years
Female-male gap life expectancy
Province, territory or nation
type==Province
type==Territory
type==Nation

dataset has changed since last saved

list place type typel-type3

1.
2.
3.
4.
5.
6.
7.
8.
9.
10 .
11.
12 .
13.

I
place
I—
I
Canada
I
Newfoundland
I
Prince Edward Island
I
Nova Scotia
I
New Brunswick
I—•
I
Quebec
I
Ontario
I
Manitoba
I
Saskatchewan
I
Alberta
I—I
British Columbia
I
Yukon
I Northwest Territories

---------------------

--------- +
type

typel

type2

Nation
Province
Province
Province
Province

0
1
1
1
1

0
0
0
0
0

Province
Province
Province
Province
Province

1
1
1
1
1

0
0
0
0
0

Province
Territory
Territory

1
0
0

0
1
1

type3 |
----------- I
1 I
0 I
0 I
0 I
0 I
—I
0 I
0 I
0 I
0 I
0 I
I
0 I
0 I
0 I

Re-expressing categorical information as a set of dummy variables involves no loss of
information, in this example, typel through type3 together tell us exactly as much as type itself

I

Data Management

37

does. Occasionally, however, analysts choose to re-express a measurement variable in
categorical or ordinal form, even though this does result in a substantial loss of information.
For example, unemp in canada2.dta gives a measure of the unemployment rate. Excluding
Canada itself from the data, we see that unemp ranges from 7% to 19.6%. with a mean of 12.26:
summarize

unemp if type

!= 3

Variable |
Obs
Mean
Std. Dev.
----------------- - -----------------------------------------------------unemp |
10
12.26
4.44877

Min

Max

7

19.6

Having Canada in the data becomes a nuisance at this point, so we drop it:
. drop if type == 3
(1 observation deleted)

Two commands create a dummy variable named unemp2 with values of 0 when
unemployment is below average (12.26), 1 when unemployment is equal to or above average
and missing when unemp is missing. In readingthe second command, recall that Stata’s sorting
and relational operators treat missing values as very large numbers.
. generate unemp2 = 0 if unemp < 12.26
(7 missing values generated)
. replace unemp2 = 1 if unemp >= 12.26
(5 real changes made)

I

&

unemp <

.

We might want to group the values of a measurement variable, thereby creating an orderedcategory or ordinal variable. The autocode function (see “Using Functions” earlier in this
chapter) provides automatic groupingof measurement variables. To create new ordinal variable
»nemp3, which groups values of unemp into three equal-width groups over the interval from
d to 20, type
. generate unemp3 = autocode(unemp,3,5,20)
(2 missing values generated)

A list of the data shows how the new dummy (unemp!) and ordinal (unemp3) variables
correspond to values of the original measurement variable unemp.
.

list place unemp unemp2 unemp3

I
place
I—
1. I
Newfoundland
2. I
Prince Edward Island
3. I
Nova Scotia
4. I
New Brunswick
5. I
Quebec
I--6. I
Ontario
7. I
Manitoba
8. I
Saskatchewan
9. I
Alberta
10 . I
British Columbia
I --11. I
Yukon
12 . I Northwest Territories
----------------------

1

unemp

unemp2

19.6
19.1
13.9
13.8
13.2

1
1
1
1
1

9.3
8.5
7
8.4
9.8

0
0
0
0
0

unemp3 I
------ 1
20 |
20 |
15 |
15 |
15 i
------ I
10 |
10 |
10 |
10 |
10 |
------ I
. I
. I

38

Statistics with Stata

Both strategies just described dealt appropriately with missing values so that Canadian
places with missing values on unemp likewise receive missing values on the variables derived
iZtmte'^' Z KerPrS,ble aPPr°aCh W°rkS b6St if °Ur data COn,ain no 'lllssin2 values To
illustrate, we begin by dropping the Yukon and Northwest Territories:
drop if unemp >= .
(2 observations deleted)

A greater-than-or-equal-to inequality such as unemp >= . will select any user-specified
missing value codes, in addition to the default code “.’’Type help missing for details
Having dropped observations with missing values, we now can use the group function
o cieate an ordinal variable not with approximately equal-width groupings, as autocode
sort the daS W grOUpingSof aPProximately equal size . We do this in two steps. First,
soil the data (assuming no missing values) on the variable of interest. Second, generate a new
vanab e using the group (#) function, where # indicates the number of groups desired. The
example below divides our 10
Canadian provinces
provinces into
10 Canadian
into 55 groups.
groups.
sort unemp

.

generate unempS = group(5)

.

list place unemp unemp2 unemp3 unempS
+ -I
I -I
I
I
I
I
I---

place

unemp

unemp2

unemp3

Saskatchewan
Alberta
Manitoba
Ontario
British Columbia

7
8.4
8.5
9.3
9.8

0
0
0
0
0

10
10
10
10
10

6.
Quebec
7. I
New Brunswick
8. I
Nova Scotia
9. I Prince Edward Island
10. I
Newfoundland

13.2
13.8
13.9
19.1
19.6

1
1
1
1
1

15
15
15
20
20

1.
2.
3.
4.
5.

unempS |
------ I
1 I
1 I
2 I
2 I
3 I
------ I
3 I
4 I
4 I
5 I
5 I

Another difference is that autocode iassigns

values equal to the upper bound of each
interval, whereas group simply assigns 1 to the first group,' 2 to the second, and so forth.

Using Explicit Subscripts with Variables
When Stata has data in memory, it also defines certain system variables that describe those data,
r examp e^
represents the total number of observations. _n represents the observation
number, n - 1 for the first observation, _n =2 for the second, and so on to the last observation
“ > r’
T 'SSUe 3 command such as *e following, it creates a new variable. caselD
equal to the number of each observation as presently sorted:
. generate caselD =

n

Sorting the data another way will change each observation’s value of_n , but its caselD value
wi lemain unchanged. Thus, if we do sort the data another way, we can later return to the
earlier order by typing

1

Data Management

.

39

sort caselD

Creating and saving unique case identification numbers that store the order of observations at
an early stage of dataset development can greatly facilitate later data management.
We can use explicit subscripts with variable names, to specify particular observation
numbers. For example, the 6th observation in dataset canadal.dta (if we have not dropped or

thousand.any

8 'S QUebeC' ConSequently’

refers t0 Quebec’s population 7334

. display pop[6]
7334.2002

Similarly, pop[I2] is the Yukon’s population:
. display pop[12]
30.1

fnmEaXPllClt subscriptin,g,and the -n system variable have additional relevance when our data
rm a series. If we had the daily stock market price of a particular stock as a variable named
puce, for instance, then either price or, equivalently. price[ _n] denotes the value of the nth
observation or day. ^-/ce[_n-l] denotes the previous day’s price, andpnce[_n+l] denoted

thTpre^ousTa^

6"

eqUa‘t0

Cha"ge

. generate difprice = price - price[_n-l]

Chapter 13, on time series analysis, returns to this topic.

Importing Data from Other Programs
i

f

Previous sections illustrated how to enter and edit data by typing into the Data Editor If our
original data reside m an appropriately formatted spreadsheet, a shortcut can speed up this
labels!7
t0 C°P'Vand P3Ste muJti-C01umnblocks data (not including column
nerh ns e y
,nt°
Data EditOr- This
some care and
vahtP cXperimentatlon> because Stata wdl interpret any column containing non-numeric
into the DateFd?'"? 3 a™
Si"Sle COlUnmS (variables>of data could also be pasted
pasted hto Fdt i°m 3
processor document. Once data have been successfully
pasted into Edito. columns, we assign variable names, labels, and so on in the usual manner.
tools^hT^rk3^’1 h metht1°dS 3re quirkbu'for largerpr0J'ects kis imPortant to have
lener^ eT
y . computerflles created by other programs. Such files fall into two
general categones: raw-data ASCII (text) files, which can be read into Stata with the
appropriate Stata commands; and system files, which must be translated to Stata format by a
special third-party program before Stata can read them.
Y
that 7 i'IUS!ra‘etASCI1 ?Ie methods’ we return t0 Ibc Canadian data of Table 2.1. Suppose
processor with a 7in?
S,ata’S
Edit°r’ We typed them int0 our word
ff thX n 7
i 0,16 Sp3Ce tWeen each Value- Strin8 values must be “ double quotes
if they contam internal spaces, as does “Prince Edward Island”. For other string values quotes
are optional. Word processors allow the option of saving documents as ASCH (text) files a
simpler and more universal type than the word processor’s usual saved-file format. We can
thus create an ASCII file named canada.raw that looks something like this:
--------

s.i

40

Statistics with Stata

*?

!:;i

’’Canada" 29606.1 10.6 "5.1 81.1
"Newfoundland" 575.4 19.6 73.9 79.8
"Prince Edward Island" 136.1
19.1 74.8
"Nova Scotia" 937.8 13.9 74.2 80.4
"New Brunswick" 760.1 13.8 74.8 80.6
"Quebec" 7334.2 13.2
13.2 “4.5 81.2
"Ontario" 11100.3 9.3
9.3 “5.5 81.1
"Manitoba" 1137.5 8.5
8.5 75*' 80.8
"Saskatchewan" 1015.6 7 75.2 81.8
"Alberta" 2747 8.4 75.5 81.4
"British Columbia" 3766 9.8 75.8 81.4
"Yukon" 30.1
.
71.3 80.4
"Northwest Territories" 65.8
.
70.2. 78

I*

II

I I

81.3

Note the use <ofperiods, not blanks, to indicate missing values for the Yukon and Northwest
Territories If the dataset should have five variables, then for every observation, exactly five
values (including periods for missing values) must exist.
inf ile reads into memory an ASCII file, such as canada.raw, in which the values are
separated by one or more whitespace characters — blanks, tabs, and newlines (carriage return
line feed, or both) — or by commas. Its basic form is
. infile variable-list using filename.raw

With purely numeric data, the variable list could be omitted, in which case Stata assigns the
names varl, var2, var3, and so forth. On the other hand, we might want to give each variable
a distinctive name. We also need to identify string variables individually. For Canada raw the
inf xle command might be
. infile str30 place pop unemp mlife flife using Canada.raw, clear
(13 observations read)

The infxle vanablelistspecifiesvariablesintheorderthattheyappearinthedatafile. The
clear option drops any current data from memory’ before reading in the new file.
If any string variables exist, their names must each be preceded by a str# statement.
str30 , for example, informs Stata that the next-named variable (place) is a string variable
with as many as 30 characters. Actually, none of the Canadian place names involve more than
21 characters, but we do not need to know that in advance. It is often easier to overestimate
s ring variable lengths. Then, once data are in memory, use compress to ensure that no
variable takes up more space than it needs. The compress command automatically changes
ail variables to their most memory-efficient storage type.

b

. compress
place was str30 now str21
. describe

Contains data
obs :
vars :
size :
variable name

place
POP
unemp

13
c

~33 (99.9% of memory free)
storage
type

str21
float
float

display
format­

%21s
%9. Og
%9. Og

value
label

variable label

Data Management
m1 i fe
flife

float
float

41

*9.0g
*9.0g

Sorted by:

We can now proceed to label variables and data as <described earlier. At any point, the
commands save Canada0 (or save Canada0, replace ) would save the new
dataset in Stata format, as file ccmciduO.dtci. The original raw-data file, Canada.raw, remains
unchanged on disk.

If our variables have non-numeric values (for example, “male” and “female”) that we want
to store as labeled numeric variables, then adding the option automatic will accomplish
this. For example, we might read in raw survey data through this infile command:
.

infile gender age income vote using survey.raw, automatic

Spreadsheet and database programs commonly write ASCII files that have only one
observation per line, with values separated by tabs or commas. To read these files into Stata,
use insheet . Its general syntax resembles that of infile , with options telling Stata
whether the data delimited by tabs, commas, or other characters. For example assuming tabdelimited data,

=
.

insheet variable-list using filename. raw,

tab

Or, assuming comma-delimited data with the first row ot the file containing variable names
(also comma-delimited).
.

insheet variable-list using filename.raw, comma names

With insheet we do not need to separately identify string variables. If we include no
variable list, and do not have variable names in the file’s first row, Stata automatically assigns
the variable names varJ. var2, var3
Errors will occur if some values in our ASCII file are
not separated by tabs, commas, or some other delimiter as specified in the insheet
command.
Raw data files created by other statistical packages can be in “fixed-column” format, where
the values are not necessarily delimited at all, but do occupy predefined column positions. Both
infile and the more specialized command infix permit Stata to read such files. In the
command syntax itself, or in a “data dictionary” existing in a separate file or as the first part of
the data file, we have to specify exactly how the columns should be read.

Here is a simple example. Data exist in an ASCII file named nfresour.raw:
193624087641691000
198725247430001044
193825138637481086
193925358964371140
1990
1991

I

8615731195
7930001262

These data concern natural resource production in Newfoundland. The four variables occupy
fixed column positions: columns 1-4 are the years (1986...1991); columns 5-8 measure
forestry production inthousands of cubic meters (2408...missing); columns 9-14 measure mine
production in thousands of dollars (764,169...793,000); and columns 15-18 are the consumer
price mdex relative to 1986 (1000...1262). Notice that in fixed-column format, unlike space
or tab-dchmited files, blanks indicate missing values, and the raw data contain no decimal
points. To read j;//-e5O7yr.ravv.intp Stata, we s pecify each variable’s column position:
___

42

Statistics with Stata

.

infix year 1-4

.

list

wood 5-8 mines 9-14
using nfresour. raw, clear
(6 observations read)

i.
2.
3.
4.
5.
6.

I year
I----I 1986
I 1987
I 1988
I 1989
I 1990
|----I 1991
+-----

wood

mines

24 03
2524
2513
2535

76416?
7430::
86374=
89643'
8615"3

CPI 15-18

CPI I
---------- 1
1000 |
1 044 |
1086 |
1140 !
1195 i

-------(

793 0-: j

1262

i

More complicated
eunipncateo fixed-column
nxecKolumn formats
formats might
might require
require a data “dictionary.” Data
dictionaries can be straightforward, but they offer many possible choices. Typing help
infix or help infileZ obtains brief outlines of these commands. For more examples
and explanation, consult the User's Guide and reference manuals. Stata also can load, write,
or view data from OBDC (Open Database Connectivity) sources: see help obdc .
What if we need to export data from Stata to some other. non-OBDC program? The
outf ile command writes ASCII files to disk. A command such as the following will create
a space-delimited ASCII file named canada6.raw, containing whatever data were in memory:
.

I5
f I: j

I

4|

outfile using canada6

The infile , insheet, infix , and outfile commands just described all
manipulate raw data in ASCII files. A second, very quick, possibility is to copy your data from
Stata s Browser and paste this directly into a spreadsheet such as Excell. Often the best option
however, is to transfer data directly between the specialized system files saved by various
spreadsheet, database, or statistical programs. Several third-party programs perform such
translations. Stat/Transfer, for example, will transfer data across many different formats
including dBASE, Excel, FoxPro, Gauss. JMP. Lotus. MATLAB, Minitab, OSIRIS, Paradox
S-Plus, SAS, SPSS, SYSTAT. and Stata. It is available through Stata Corporation
(www.stata.com) or from its maker. Circle Systems (wvvw_stattransfer.com). Transfer prosrams
prove indispensable for analysts working in multi-program environments or exchanging data
with colleagues.

Combining Two or More Stata Files

1

i

We can combine Stata datasets in two general ways: append a second dataset that contains
additional observations; or merge with other datasets that contain new variables or values.
In keeping with this chapter s Canadian theme, we will illustrate these procedures using data
on Newfoundland. File newfl.dta records the province’s population for 1985 to 1989.

r

Data Management
. use newfl, clear
(Newfoundland 1955-89)

.

describe

Contains data from C:\data\newfl.dta
obs:
5
Newfoundland 1985-89
vars:
2
3 Jul 2005 10:49
s^ze:
50
(99.9% of memoryfree)
variable name

year
pop

storage
type

display
format

ir.t
float

%9. Og
%9. Og

value
label

variable label
Year
Population

Sorted 'ey:

.

list

1.
2.
3.
4.
5.

cop |
------ I
58C~00 |
58C200 ;
56=200 |
565100 |
57C300 |

File nevvf2.dta has population and unemployment counts for some later years:

f
I

I year
I
I 1985
I 1986
I 1987
I 1 988
I 1989

. use newf2
(Newfoundland 1991-95)

describe

Contains data from C:\data\newf2.dta
obs:
6
vars:
3
size:
84 (99.9% of memory free)
variable name

year
pop
j obless

storage
type

display
format

float
float

%9. Og
%9. Og
%9. Og

value
label

Newfoundland 1990-95
3 Jul 2005 10:49

variable label

Year
Population
Number of people unemployed

Sorted by:

. list

1.
2.
3.
4.
5.

6.

I year
I----------

cop

I 1990
I 1991
I 1992
I 1993
I 1994
I
I 1995
+----------

573400
573500
575600
584400
582400
575449

------- +
jobless |
I
42000 I
45000 I
49000 I
49000 I
50000 I
I
I

To combine these datasets, with newf2.dta already in memory, we use the append command:
append using newfl

43

I

44

Statistics with Stata

!
list

3
4
5

I year
I----! 1990
' 1991
I 1992
I 1993
I 1994

6
7
8
10

11 .

1995
1985
1986
198 7
1988

I
I
I 1989
+-----

573400

575600
584403
582400

4 2 o ■:: I
4 50 0 0 I
4 9000 I
4 900 3 I
5000 0 I
I

575449
580700
5Q 9 - '
568000
570000

Because variableyWew occurs in newf2( 1990 to 1995) but not in newfl, its 1985 to 1989
values are
are imissing in the combmed dataset. We can now put the observations in order from
eai liest to latest and save these combined
data as a new file, newfo.dtcr.
------------sort year

.

list

3
4
5

6

8
9
10
11 .

I year

pop

jobless |
-------- I

■ 1985
i 1986
I 1987
I 1988
I 1989

580700
580200
568200
568000
570000

• I
• I
• I
. I
. I
-------- I

I 1990
! 1991
i 1992
i 19 93
I 1994
i----! 1995

573400
573500
575600
584400
5824 00

42000 |
45000 !
49000 |
49000 |
50000 |
-------- (

575449

save newf3

append
might seco^X^vJLXZva^
be compared to lengthening
memoiy)
by tapinga

urther Newfoundland time series: the numbers of births and divorces over the years 1980 to
' SA°me °bservatlons in common with our earlier dataset newfS.dta, as well as
one
e (yeai) in common, but it also has two new variables not present in newfl.dta.
. use newf4
(Newfoundland 1980-94)

describe

Contains data from C:\data\newf4.dta
obs :
15
vars :
3
size:
150 (99.9% of memory free)

Ji

Newfoundland 1980-94
3 Jul 2005 10:49

Data Management

name

year
births
divorces

storage
type

int
int
int

display
format

value
label

45

variable label

%9.0g
%9.0g
%9.0g

Year
Number of births
Number of divorces

Sorted by:

list

------------- +
1.
2.
3.
4.
5.

year

births

1980

10332
11310
9173
9630
8560

I 1981
I

1982

6.

I 1983
I 1984
I
I 1985

3.
9.
10.

1986
1987
1988
1989

11 .
12 .
13.
14 .
15 .

I
I
I
I
I 1990
I 1991
I 1992
I 1993
I 1994

8080
8320
7656
7396
7996

7354
6929
6689
6360
6295

divorces |
-I
555 I
569 I
625 I
711 I
590 I
I
561 I
610 I
1002 I
884 I
981 I
I
973 I
912 I
8 67 I
930 I
933 I

We want to merge new/3 with newf4, matching observations according toyear wherever
possib e. To accomplish this, both datasets must be sorted by the index variable (which in this
example is year). We earlier issued a sort year command before saving newfS.dta, so we
now do the same with newf4.dta. Then we merge the two, spccifyingyear as the index variable
to match.
sort year

l
.

merge year using newf3
describe

Contains data from newf4.dta
obs:
16
vars:
6
size:
304 (99.9% of memory free)

variable name
year
births
divorces
POP
j obless
_me rge

Sorted by:
Note :

I

list

storage
type
int
int
int
float
float
byte

display
format

% 9.0g
%9.0g
% 9.0g
%9.0g
%9.0g
%8.0g

value
label

Newfoundland 1980-94
3 Jul 2005 10:49

variable label

Year
Number of births
Number of divorces
Population
Number of people unemployed

dataset has changed since last saved

r

46

Statistics with Stata

o' i •' r r e s
i.
2

6
7
3
9
10
11
12
13
14
15

16.

i
I

198 0
1981

I
I

1983
1984

pop

jobless

merge |
------------ I
1 I
1 I
1 I
1 I
1 I

569
625
"11
590

----------- !
I 1987
i 198 6
1 1 89
I---------I 1990
! 1991
I 1992
1 19 93
i 1994

! 1995
+----------

765€

"3 = 4
6 92 9
66 6 9
62 ? 3

561
610
10 02
= 84
951

580700
580200
568200
568000
570000

9"3
912
= 67
930
933

573400
573500
575600
584400
582400

3 I
3 I
3 I
3 I
3 I
------------ I

42000
45000
4 9000
49000
50000

575449

3 I
3 I
3 I
3 I
3 I
----------- I

2

I

--------- +

ipnXd S
h
e a ready ln memory> are retained and those of the “using” data are
gnored The merge command has several options, however, that override this default A
6 °ll0'Vlng form would allow any hissing values in the master data to be
replaced by corresponding nonmissing values found in the using data (here, neyvfS.dtdy.
. merge year using newfS, update

™n^«mman1 SUC5 aS thu foIIowing causes my values from the master data to be replaced by
nonmissing values from the using data, if the latter are different:
Y
. merge year using nevf5, update replace

examnST6
VariabIe °CCUr m0re than once in the master data; for
= . WO
PP°!e 'hat tJe yey 1990 Occurs twice- Then values from the using data wither
b
matchedwi ‘h each occurrence of year= 1990 in the master data. You can use this
dam on a5
SUch as combining background data on individual patients with
taon ^ny number of separate doctor ^‘5 theymade Aithough merge makes this and
many other data-management tasks straightforward, analysts should look closely at the results
o be certain that the command is accomplishing what they intend.
As a diagnostic aid, merge automatically creates a new variable called
merge. Unless
update was specified, _merge codes have the following meanings:

1
2
3

Observation from the master dataset only.
Observation from the using dataset only.
Observation from both master and using data (using values ignored if different).

i

Data Management

47

If the update option was specified, merge codes convey what happened:
1 Observation from the master dataset only.
2

Observation from the using dataset only.
Observation from both, master data agrees with using.
Observation from both, master data updated if missing.
Observation from both, master data replaced if different.
Before performing another merge operation, it will be necessary to discard or rename this
variable. For example,
3
4
5

.

drop _merge

Or,
rename _merge __mergel

I
I

We can merge multiple datasets with a single merge command. For example, if newf5.dta
through newfS.dta are four datasets, each sorted by the variable year, then merging all four with
the master dataset could be accomplished as follows.
. merge year using newfS newf6 newf7 newfB,

update replace

Other merge options include checks on whether the merging-variable values are unique, and
theabilitytospecify which variables to keep for the final dataset. Type help merge for
details.

I

Transposing, Reshaping, or Collapsing Data
Long after a dataset has been created, we might discox er that for some analytical purposes it
has the wrong organization. Fortunately. se\ eral commands facilitate drastic restructurins of
datasets. We will illustrate these using data (grow thl.dta} on recent population growth in five
eastern provinces of Canada. In these data, unlike our prexious examples, province names are
represented by a numerical variable with eight-character labels.
. use growthl, clear
(Eastern Canada growth)
describe

Contains data frorn C:\data\growthl.dta
obs:
5
vars:
5
size:
105 (99.9% of memory free)

I

variable name
provinc2
grow92
grow93
grow94
grow95
Sorted by:

storage
tvce
byte
float
float
float
float

display
format

value
label

%8 . Og
%9 . Og
%9. Og
%9.0g
%9.0g

provinc2

Eastern Canada growth
3 Jul 2005 10:48

variable label
Eastern Canadian province
Pop. gain in 1000s, 1991- 92
Pop. gain in 1000s, 1992- 93
Pop-, gain in 1000s, 1993- 94
Pop. gain in 1000s, 1994- 95

1
48

Statistics with Stata

list

1.
2.
3.
4.
5.

+-------I provinc2
|-------I New Brun
I Newfound
I Nova Sco
I Ontario
Quebec
I
+

grow92

grow93

grow94

grow 9 5

10
4.5
12.1
174.9
80. 6

2.5
.8
5.8
169.1
77.4

2.2
-3
3.5
120.9
48.5

I
2.4
-5.8
3.9 I
163.9 I
47.1 I

In this organization, population growth for each year is stored as a separate variable. We
could analyze changes in the mean or variation of population growth from year to year. On the
other hand, given this organization, Stata could not readily draw a simple time plot of
population growth against year, nor can Stata find the correlation between population growth
m New Brunswick and Newfoundland. All the necessary information is here, but such analyses
require different organizations of the data.
One simple reorganization involves transposing variables and observations. In effect the
dataset rows become its columns, and vice versa. This is accomplished by the xpose
command. The option clear is required with this command, because it always clears the
present data from memory. Including the varname option creates an additional variable
(named _yarname ) in the transposed dataset, containing original variable names as strings.

I

. xpose, clear varname
describe

Contains data
obs :
vars :
size:
variable name

vl
v2
v3
v4
v5
varname
Sorted by:
Note:

5
6
160 (99.9% of memory free)

storage
type
float
float
float
float
float
str8

display
format

value
label

variable label

%9.0g
%9.0g
%9.0g
%9. Og
%9. Og
%9s

i

dataset has changed since last saved

list

vl
I
I
1. I
1
2. I
10
3 . I 2.5
4 . I 2.2
5 . I 2.4
+--------

v2

v3

v4

v5

2
4.5
.8
-3
-5.8

3
12.1
5.8
3.5
3.9

4
174.9
169.1
120.9
163.9-

5
80.6
77.4
48.5
47 . 1

-------- +
_varname I
---------- i

provincz f
grow92 |
grow93 I
grow94 |
grow95 |
-------- +

alue labels are lost along the way, so provinces in the transposed dataset are indicated
only by their numbers (1 = New Brunswick, 2 = Newfoundland, and so on). The second
through last values in each column are the population gainslor that province, in thousands.

Data Management

49

Thus, variable vl has a province identification number (1, meaning New Brunswick) in its first
row, and New Brunswick’s population growth values for 1992 to 1995 in its second throueh
fifth rows. We can nowfind correlations between population growth in different provinces, for
instance, by typing a correlate command with in 2/5 (second throueh fifth
observations only) qualifier:
. correlate vl-v5 in 2/5
(obs=4)

vl |
v2 |
v3|
v4 |
v5|

V1

v2

v3

v4

v5

1 .0000
0.8058
0.9742
0.5070
0.6526

1.0000
0.8978
0.4803
0.9362

1.0000
0.6204
0.8049

1.0000
0.6765

1.0000

The strongest correlation appears between the growth of neighboring maritime provinces New
Brunswick (vl) and Nova Scotia (v3): >■ = .9742. Newfoundland’s (v2) growth has a much
weaker correlation with that of Ontario (y4)\ r = .4803.
More sophisticated restructuring is possible through the reshape command. This
command switches datasets between two basic configurations termed “wide” and “lon^ ”
Dataset growthl.dta is initially in wide format.

I

. use growthl, clear
(Eastern Canada growth)
.

list

I provinc2
I-------1 . I New Brun
2 . I Newfound
3. I Nova Sco
4. I
Ontario
5. I
Quebec

grow92

grow93

grow94

10
4.5
12.1
174.9
80.6

2.5
.8
5.8
169.1
77.4

2.2
-3
3.5
120.9
48.5

grow95 I
------------- I
2.4 j
-5.8 |
3.3 i
163.9 i
47.1 1

A reshape command switches this to long format.
reshape long grow, i(provinc2)
(note: j = 92 93 94 95)
Data

j(year)

wide

Number of obs.
5
Number of variables
5
j variable (4 values)
xij variables:
grow92 grow93 ... grow95

I
I

..J

long

20
3
year
grow

Listing the data shows how they were reshaped. A sepby () option with the list
command produces a table with horizontal lines visually separating the provinces, instead of
every five observations (the default).

50

.

Statistics with Stata

list.

sepby(provinc2)

I provinc2

yea r

g rc w

New Brun
New Brun
New Brun
New Brun

92
93
94
95

io

I

2.5
2.2

|
j

I Newfound
I Newfound
I Newfound
I Newfound
I-----------------9 . I Nova Sco
10 . I Nova Sco
11 . I Nova Sco
12 . I Nova Sco
I
13 . I
Ontario
14 . I
Ontario
15 . I
Ontario
16. I
Ontario
I
17 . I
Quebec
18 . I
Quebec

92
93
94
95

I
I
I
I
I
I

1.
2.
3.
4.

5.
6.
7.
8.

19 .
20 .

I
I

Quebec
Quebec

92
93
94
95
92
93
94
95
92
93
94
95

2.4

|
I
4.5 I
.8 I
-3 I
-5.8 I
I
12.1
5.8 I
3.5 I
3.9 I
I
174 . 9 I
169.1 I
120.9 I
163.9 I
I
80.6 I
77.4 I
48.5
47.1

.

label data "Eastern Canadian growth--long"

.

label variable grow "Population growth in 1000s"

.

save growth2

file C:\data\growth2.dta saved

The reshape command above began by stating that we want to put the dataset in long
form. Next, it named the new variable to be created, grow. The i (provinc2) option
specified the observation identifier, or the variable whose unique values denote logical
observations. In this example, each province forms a logical observation. The j (year)
option specifies the sub-observation identifier, or the variable whose unique values ( within each
logical observation) denote sub-observations. Here, the sub-obsen ations are vears within each
province.
Figure 2.1 shows a possible use for the long-format dataset. With one graph command,
we can now produce time plots comparing the population gains in New Brunswick.
Newfoundland, and Nova Scotia (observations for which pro\inc2 < 4). The graph
command on the following page calls for connected-line plots of grow (as r-axis variable)
against year (x axis) ifprovince? < 4. with horizontal lines at v = 0 (zero population growth),
and separate plots for each value ofprovinc?.

I

Data Management

graph twoway connected grow year if provinc2 < 4,
by(provinc2)



New Brun

51

yline(O)

Figure 2.1

Newfound

o

o
o o
o

.E “J

92

O
O)

93

94

95

Nova Sco

c
•° o
ro T~
Z5
Q. IO
O
CL
O

“? -

92

93

94

95

year
Graphs by Eastern Canadian province

ec meh in their fisheries during the early 1990s contributed to economic hardships in these
three provinces. Growth slowed dramatically in New Brunswick and Nova Scotia, while
Newfoundland (the most fisheries-dependent province) actually lost population.
reshape works equally well in reverse, to switch data from “long” to “wide” format
Dataset growths.dta serves as an example of long format.
.

use growths,

clear

(Eastern Canadian grcwth--lcng)
list.

sepby(provinc2)

!

------- +
1.
2.
3.

4.
5.
6.
7.
8.

9.
10 .
11 .
12 .
13 .
14 .
15.
16.

provincz

grow

i New Brun
i New Brun
I New Brun
New Brun

10
2.5
2.2
2.4

Newfound
Newfound
I Newfound
I Newfound
I--------I Nova Sco
I Nova Sco
I Nova Sco
I Nova Sco
I
I
Ontario
I
Ontario
Ontario
I
I
Ontario
I

4.5

.=
-5.S

year I
--------- I
92 I
93

|

94
95

|
I

92 |
93 i
94 I
95 I
■I

12.1
5.8
3.5

92
93

3.9

94
95

174.9
169.1
120.9
163.9

92
93
94
95

I
I
I
I
I
I
I
I
I
I



V

-1 CTO

1009

9

A

!

.<G

' - li

52

Statistics with Stata

Quebec
Quebec
Quebec
Quebec

19.
20 .

8 0.6
77.4
48.5
47.1

92
93
94
95

To convert this to wide format, we use reshape wide :
reshape wide

(note:

grow,

i(provinc2)

j(year)

= 92 93 94 95)

Data

wide

Number of obs.
Number of variables
j variable (4 values)
xij variables:

20
3
year

5
5
(dropped)

grow

grow92 grow93 ... grow95

list
I provinc2
I
1 . I New Brun
2 . I Newfound
3. I Nova Sco
4. I
Ontario
5. I
Quebec
+

grow92

grow93

grow94

grow95

I
■ I

10
4.5
12. 1
174.9
80.6

5
8
5.8
169.1
7’ . 4

2.2
-3
3.5
120.9
48.5

2.4
-5.8
3.9
163.9
47.1

I
I
I
I
I

+

Notice that we have recreated the organization of dataset growthl.dta.
Another important tool for restructuring datasets is the collapse command, which
creates an aggregated dataset of statistics (for example, means, medians, or sums). The long
gi o\vth3 dataset has four observations for each province.*
. use growths, clear
(Eastern Canadian growth—long)

list,

.

sepby (provinc2)

I provi.-.cz

grow

year

1.
2.
3.
4.

New Brun
New Brun
New Brun
New Brun

10
2.5
2.2
2.4

92
93
94

5.
6.

! Newfound
I Newfound
I Newfound
1 Newfound

4.5
.8
-3
-5.8

93
94
95

I N ova Sco
I Nova Sco
I Nova Sco
I Nova Sco
I
I
Ontario
Ontario
I
Ontario
I
I
Ontario

12.1
5.8
3.5
3.9

92
93
94
95

174.9
169.1
120.9
16 3.9

92
93
94
95

•7

8.
9.
10 .
11 .
12 .

13.
14 .
15 .
16.

|
j
I
I
I
I

I

Data Management

17.
18.
19.
20.

I
|
|
|
|

Quebec
Quebec
Quebec
Quebec

80.6
77.4
48.5
47.1

------ 1
92 |
93 |
94 |
95 |

that is, one province.
.

collapse

(mean)

y

'

1 idoie,

grow, by (provino2)

list
I provinc2

i

|------

i. I New Brun
2 . I Newfound
3 . I Nova Sco
4. I
Ontario
5. I
Quebec
+

grow |
--------- I
4.275 |
-.8750001 |
6.325 |
157.2 |
63.4 |

I

I

variable name as wrtb grow in the previous example, or births and deaths, the collapsed
variable takes on the same name as the old variable.
i-onapsea
.

collapse (sum) births deaths (mean) meaninc =
income
(median) medinc = income, by(provinc2)

collapse can create variables based on the following summary statistics:
mean
sd
sum

rawsum
count

max
min

median

Pl
P2

iqr

53

Means (the default; used if the type of statistic is not specified)
Standard deviations
Sums

Sums ignoring optionally specified weight
Number of nonmissing observations
Maximums
Minimums
Medians
1st percentiles
2nd percentiles (and so forth to P9 9 )
Interquartile ranges

Statistics with Stata

54

Weighting Observations

Stata understands four types of weighting:
aweight Analytical weights, used in weighted least squares (WLS) regression and similar
procedures.
fweight

Frequency weights, counting the number of duplicated observations. Frequency
weights must be integers.
iweight Importance weights, however you define “importance.”
pweight Probability or sampling weights, equal to the inverse of the probability that an
observation is included due to sampling strategy.
Researchers sometimes speak of “weighted data.” This might mean that the original sampling
scheme selected observations in a deliberately disproportionate way, as reflected by weights
equal to 1/(probability of selection). Appropriate use of pweight can compensate for
disproportionate sampling in certain analyses. On the other hand, “weighted data” might mean
something different
an aggregate dataset, perhaps constructed from a frequency table or
cross-tabulation, with one or more variables indicating how many times a particular value or
combination of values occurred. In that case, we need fweight.
Not all types of weighting have been defined for all types of analyses. We cannot, for
example, use pweight with the tabulate command. Using weights in any analysis
requires a clear understanding of what we want weighting to accomplish in that particular
analysis. The weights themselves can be any variable in the dataset.
The following small dataset (nfschool.dta\ containing results from a survey of 1,381 rural
Newfoundland high school students, illustrates a simple application of frequency weighting,
describe
Contains data from C:\data\nfschool.dta
obs:
6
vars:
3
size:
48 (99.9% of memory free)
storage
type

display
format

value
label

byte
byte
int

%8.0g
%8.0g
%8.0g

yes

list, sep(3)
+----------------I univers
year
I
1. I
no
10
2. I
no
11

count

variable name
univers
year
count

Sorted by:

3.

I
I

no

12

210
260
274

4.
5.
6.

I
I
I

yes
yes
yes

10
11
12

224
235
178

Newf.school/univer.(Seyfrit 93)
3 Jul 2005 10:50

variable label
Expect to attend university?
What year of school now?
observed frequency

Data Management

55

At first glance, the dataset seems to contain only 6 observations, and when we cross­
tabulate whether students expect to attend a university (univers) by their current year in high
school (year), we get a table with one observation per cell.
tabulate univers year

Expect to
attend
university
?

|
I
|
I

What year of school now?
10
11
12 |

Total

—+
no |
yes |

1
1

1
1

1 I
1 I

3
3

Total |

2

2

2 I

6

To understand these data, we need to apply frequency weights. The variable count gives
frequencies: 210 of these students are tenth graders who said they did not expect to attend a
university, 260 are eleventh graders who said no, and so on. Specifying [fweight =
count] obtains a cross-tabulation showing responses of all 1,381 students.
tabulate

Expect to
attend
university
?

|
|
I
I

univers year

[fweight = count]

What year of school now?
10
11
12 |

------ +

Total

no I
yes I

210
224

260
235

274 |
178 |

744
637

Total |

434

495

452 |

1, 381

Carrying the analysis further, we might add options asking for a table with column
percentages ( col ), no cell frequencies ( nof), and a %2 test of independence ( chi2 ). This
reveals a statistically significant relationship (P = .001). The percentage of students expecting
to go to college declines with each year of high school.
tabulate univers year [fw = count],
Expect to
attend
university
?

|
I
|
I

What year of school now?
10
11
12 |

Total

no |
yes |

48.39
51.61

52.53
47.47

60.62 |
39.38 |

53.87
46.13

Total |

100.00

100.00

100.00 |

100.00

Pearson chi2(2) =

i

col nof chi2

13.8967

Pr = 0.001

Survey data often reflect complex sampling designs, based on one or more of the following:
disproportionate sampling— for example, oversampling particular subpopulations, in order
to get enough cases to draw conclusions about them.
clustering — for example, selecting voting precincts at random, and then sampling individuals
within the selected precincts.

56

Statistics with Stata

strattfcation
tor example, dividing precincts into “urban” and “rural” strata, and then
sampling precincts and/or individuals within each stratum.
Complex sampling designs require specialized analytical tools, pweights and Stata’s
ordinary analytical commands do not suffice.
Stata’s procedures for complex survey data include special tabulation, means, regression
ogit probit, tobit and Poisson regression commands. Before applying these commands, users
must first set up their data by identifying variables that indicate the PSUs (primary sampling
units) or clusters, strata, finite population correction, and probability weights. This is
accomplished through the svyset command. For example:
.

svyset precinct [pweight=invPsel], strata(urb_rur)

fpc(finite)

For each observation in this example, the value of variablepnnczncZ identifies PSU or cluster
\ alues of urb rur identify the strata,finite gives the finite population correction, and invPsel
gives the probability weight or inverseofthe probability of selection. After the data have been
svyset and saved, the survey analytical procedures are relatively straightforward.
Commands
—3 arc typically prefixed by svy: , as in
svy:

mean income

or
svy.

regress income education experience gender

The Survey Data Reference Manual contains full details and examples of Stata’s extensive
survey-analysis capabilities. For online guidance, type help svy and follow the links to
particular commands.

Creating Random Data and Random Samples
The pseudo-random number function uniform () lies at the heart of Stata’s ability to
generate random data or to sample randomly from the data at hand. The Base Reference
Manual (Functions) provides a technical description of this 32-bit pseudo-random generator.
It we presently have data m memory, then a command such as the following creates a new
variable named randnum, having apparently random 16-digit values over the interval (0 1) for
each case in the data.

. generate randnum = uniform()

Altei natively, we might create a random dataset from scratch. Suppose we want to start a
new dataset containing 10 random values. We first clear any other data from memory (if they
were valuable, save them first). Next, set the number of observations desired for the new
dataset. ^Explicitly setting the seed number makes it possible to later reproduce the same
random results. Finally, we generate our random variable.
.

clear

set obs 10
obs was 0, now 10
.

set seed 12345

.

generate randnum = uniform()

I
i

Data Management

57

■-

list
+-

randnum
I
I1. I
.309106
2 . I .6852276
3 . I .1277815
4 . I .5617244
5 . I .3134516
I-6. I ..5047374
7 . I ■.7232868
8. I •.4176817
9. I ..6768828
10 . I ..3657581

I
I
I
I
I
I
I
I
I
I
I
I
I

In combination with Stata’s algebraic, statistical, and special functions, uniform () can
simulate values sampled from a variety of theoretical distributions. If we want newvar sampled
from a uniform distribution over [0,428) instead of the usual [0,1), we type
. generate newvar = 428 * uniforrnQ

These will still be 16-digit values. Perhaps we want only integers from 1 to 428 (inclusive):

I

.

generate newvar = 1 + trunc(428

.

clear

* uniform())

To simulate 1,000 rolls of a six-sided die, type
. set obs 1000
obs was 0, now 1000

generate roll — 1 + trunc(6 * uniforrnQ)
. tabulate roll
die |

Freq.

1 I
2 I
3 I
4 I
5 I
6 I
—+
Total |

171
164
150
170
169

1000

Percent

Cum.

16.40
15.00
17.00
15.90
17.50

33.50
48.50
4-j

100.00

We might theoretically expect 16.67% ones, 16.67% twos, and so on, but in any one sample like
these 1,000 “rolls,” the observed percentages will vary randomly around their expected values.
To simulate 1,000 rolls of a pair of six-sided dice, type

I

.

generate dice = 2 + trunc(6 * uniform())

.

tabulate dice
dice |

Freq.

Percent

Cum.

2
3
4
5
6
7
.8

26
62
78
120
153
14 9
. 146

2.60
6.20
7.80
12.00

2.60
8.80 16.60
28.60
43.90
58.80
73.40

------ +

1

I
I
I
I
I
I
I

15.30

14.90

.

___

+ trunc(6 * uniform())

58

Statistics with Stata

1!

9 I
10 |
11 I
12---|

96
88
53
29

9. 60
8.80
5.30
2.90

Total |

1000

100.00

-------- +

■I i

83.00
91.80
97.10
100.00

We can use _n to begin an artificial dataset as well. The following commands create a new
5,000-observation--------------dataset with1 one variable named index, containing values from 1 to 5,000.
. set obs 5000
obs was 0, now 5000

.

generate index =

n

summarize

|

Obs

Mean

Std. Dev.

Min

Max

index |

5000

2500.5

1443.52

1

5000

Variable

It is possible to generate variables from a inormal• (Gaussian)
‘ ‘ distribution
_I‘
using
uniform(). The following example creates a dataset with 2,000 observations and 2
variables, z from an N(0,l) population, andx from N(500,75).
.

clear

. set obs 2000
obs was 0, now 2000

. generate z = invnormal(uniform())
.

generate x = 500 + 75*invnormal(uniform () )

The actual sample means and standard deviations differ slightly from their theoretical values:
summarize
Variable |
------------------------- +
.

f

z
X

I
I

Obs

Mean

Std. Dev.

Min

Max

2000
2000

.0375032
503.322

1.026784
75.68551

-3.53 62 09
244.33=4

4.038878
7 4 3.13 •’7

If 2 Allows a normal distribution, v - e' follows a lognormal distribution. To form a
lognormal variable v based upon a standard normal z,
. generate v = exp(invnormal(uniform())

To form a lognormal variable tv based on an N(100,15) distribution,
generate w = exp(100 + 15*invnormal(uniform())

Taking logarithms, of course, normalizes a lognormal variable.
To simulate y values drawn randomly from an exponential distribution with mean and
standard deviation |i = o = 3,
.

generate y = -3

*

In (uniform())

For other means and standard deviations, substitute other values for 3.
XI follows a %2 distribution with one degree of freedom, which is the same as a squared
standard normal:

I

. generate XI = (invnormal(uniform())A2

By similar logic, X2 follows a yj with two degrees of freedom:

Data Management

generate X2 -

.

(invnormal(uniformO))^2 +

59

(invnormal(uniform()))-2

Other statistical distributions, including t and F. can be simulated along the same lines. In
addition, programs have been written for Stata to generate random" samples foliowine
distributions such as binomial, Poisson, gamma, and inverse Gaussian.

Although invnormal (uniform() ) can be adjusted to yield normal variates with
particular correlations, a much easierway to do this is through the drawnorm command. To
generate 5,000 observations from N(0.1). type
.

clear

.

drawnorm z,

n(5000)

summ
Variable |

Obs

Mean

I

5000

-.0005951

z

Std.

Dev.

1.019788

Min

-4.518918

3.923464

vana )les to have the following population correlations:

Xl

x2

x3

Xl

1.0

0.4

-0.8

x2

0.4

1.0

0.0

x3

-0.8

0.0

1.0

The procedure for creating such data re<■quires first defining the correlation matrix C, and then
using C in the drawnorm command:
.

mat C =

.

drawnorm xl x2 x3,

(1/

.4,

-.8

\

.4,

1,

0

\ -.8,

means (0,100,500)

0,

1)

sds(1,15,75)

corr(C)

summarize xl-x3

Variable |

Obs

Mean

;
|
|

5000
5000
5000

100.1826
500.7747

xl
x2
x3

. 0024364

Stn.

Dev.

Min

Max

• .01646
14.91325
76.93925

-3.478467
46.13897
211.5596

3.598916
150.7634
"69.6C~4

. correlate xl-x3
(obs=5000)
xl

I
I

I

xl

:

q q r. r,

x2
x3

|
|

•3951
.8134

x2

x3

1.0000
-0.0072

1.0000

Compare the sample variables ’ correlations and means with the theoretical values given earlier
Random data generated in this fashion can be viewed as samples drawn from theoretical
populations We should not expect the samples to have exactly the theoretical population
parameters (in this example, an x3 mean of500, xl-x2 correlation of0.4,x/-x5 correlation of
-.8, and so forth).

60

Statistics with Stata

The command sample makes unobtrusive use of uniform’s random generator to
obtain random samples of the data iiin memory-. For example, to discard all but a 10% random
sample of the original data, type
. sample 10

When we add an in or if qualifier, sample applies only to those observations meeting
our criteria. For example,
.

sample 10 if age <26

would leave us with a 10% sample of those observations vyith age less than 26. plus 100% of
the original observations with age >26.
We could also select random samples of a particular size, To discard all but 90 randomlyselected observations from the dataset in memory, type
.

sample 90,

count

The sections in Chapter 14 on 1bootstrapping and Monte Carlo simulations provide further
examples of random sampling and random variable generation.

Writing Programs for Data Management
Data management on larger projects often involves repetitive or error-prone tasks that are best
handled by writing specialized Stata programs. Advanced programming can become very
technical, but we can also begin by writing simple programs that consist of nothing more than
a sequence of Stata commands, typed and saved as an ASCH file. ASCII files can be created
using your favorite word processor or text editor, which should offer “ASCII text file” among
its options under File - Save As. An even easier way to create such text files is through Stata’s
Do-file Editor, which is brought up by clicking Window - Do-file Editor or the icon <$.
Alternatively, bring up the Do-file Editor by typing the command doedxt, or doedit
filename iffilename exists.
For example, using the Do-file Editor we might create a file named canada.do (which
contains the commands to read in a raw data file named canada.raw). then label the dataset and
its variables, compress it, and save it in Stata format. The commands in this file are identical
to those seen
earlier----when
---------_.i we went through the example step by step.
infile str30 place pop
p— unemp
—- . - flife
-- - - using Canada.raw
mlife

label data "Canadian dataset 1"
label variable pop "Population in 1000s, 1 995"
label variable unemp "% 15+ population unemployed,
1 9 9 5"
label variable mlife "Male life expectancy years"
label variable
"Female li e expectancy years"
compres s
save canadal, replace

Once this canada.do file has been written and saved, simply typing the following command
causes Stata to read the file and run each command in turn:
. do Canada

Data Management

61

Such batch-mode programs, tenned “do-files,” are usually saved with a .do extension. More
elaborate programs (defined by do-files or“automatic do” files) can be stored in memory, and
can call other programs in turn — creating new Stata commands and opening worlds of
possibility tor adventurous analysts. The Do-file Editor has several other features that you
might find useful. Chapter 3 describes a simple way to use do-files in building graphs. For
further information, see the Getting Started manual on Using the Do-file Editor.
Stata ordinarily interprets the end of a command line as the end of that command. This is
reasonable onscreen, where the line can be arbitrarily long, but doesnot work as well when we
are typing commands in a text file. One way to avoid line-length problems is through the
#del imi t command, which can set some other character as the end-of-command delimiter.
In the following example, we make a semicolon the delimiter; then type two long commands
that do not end unti I a semicolon appears; and then finally reset the delimiter to its usual value,
a carriage return (cr):
#delimit ;
in.Li.Le str30 place pop unemp mlife flife births deaths
marriage medinc mededuc using newcan.raw;
order place pop births deaths marriage medinc mededuc
unemp mlife flife;
ffdelimit cr

Ii

Stata normally pauses each time the Results window becomes full of information, and waits
to proceed until we press any key (or Qj). Instead of pausing, we can ask Stata to continue
scrolling until the output is complete. Typed in the Command window or as part of a program
the command


.

I

set more off

calls for continuous scrolling. This is convenient if our program produces much screen output
that we don’t want to see, or if it is writing to a log file that we will examine later. Typing
set more on

returns to the usual mode of waiting for keyboard input before scrolling.

Managing Memory
When we use or File - Open a dataset, Stata reads the disk file and loads it into memoiy.
Loading the data into memory permits rapid analysis, but it is only possible if the dataset can
fit within the amount of memory currently allocated to Stata. If we try to open a dataset that
is too large, we get an elaborate error message saying “no room to add more observations,” and
advising what to do next.

I

. use C:\data\gbank2.dta

(Scientific surveys off S. Newfoundland)
no room to add more observations
An attempt was made to increase the number of observations beyond what is
currently possible. You have the following alternatives:
1. Store your variables more efficiently; see help compress.
(Think of
State's data area as the area of a rectangle; Stata can trade off width
and length.)

2.

Drop some variables or observations; see help drop.

r

62

Statistics with Stata

3.

Increase the amount of memory allocated to the data
area using the set
memory command; see help memory.

r( 901) ;

Sma 1 Stata allocates a fixed amount of memory to data, and this limit cannot be changed.
Intercooled Stata and Stata/SE versions are flexible, however. Default allocations equal 1
megabyte for Intercooled, and 10 megabytes for Stata/SE. If we have Intercooled or Stata/SE,
running on a,computer with enough physical memory, we can set Stata’s memory allocation
higher
with the-----setmemory command. To allocate 20 megabytes to data, type

.

set memory 20m

Current memory allocation
current
value

set table

set maxvar
set memory
set matsize

5000
2 0M
400

memory usage
(IM = 1024k)

description

max. variables allowed
max. data space
RHS vars in models

1.733M
20.000M
1.254M
22.987M

If there are data already in memory, first type the command clear to remove them. To reset
the memory allocation “permanently," so it will be the same next time we start up, type
set memory 20m, permanently

In the example given earlier, gbcmk2.dta is a 11.3-megabyte dataset that would not fit into
the defaul t allocation. Asking for a 20-megabyte allocation has now given us more than enough
room for these data.

I

Contains data from C:\data\gbank2.dta
obs:
74,078
var s :
size:

v a r i a b 1 e n =me



r=c_typc
vessel
trip
set
rank
assembla
yea r
month
day
set_type
stratum
division
unit_are
light
wind_dir
wind_for
sea
bottom
t ime_mid
duration
tow dist

Spring scientific surveys NAFO
3KLNOPQ, 1971-93
2 Mar 2000 21:28

(46.0% of memory free)

storage
type

byte
int
int
str?
byte
byte
byte
byte
int
str2
str 3
int
byte
byte
byte
byte
int
byte
int

display
format

%9.0g
% 4.0 g
%4.0g
%8.0g
%8.0g
%8.0g
%7s
* 4.0g
%4.0g
% 4.0g
%8.0g
%8.0g
%2s
%3s
%8.0g
%4.0g %4.0g
%4 . Og
%4.0g
%8.0g
%8.0g
%8.0g

value
label

variable label
original case number
Vessel
Trip number
Set number

set_type

Year
Month
Day
Set type
Stratum or line fished
NAFO division
Nfld. area grid map square
Light conditions
Wind direction

Wind force
Type of bottom
Time (midpoint)
Duration of set
Distance towed

r

Data Management
gear_oy
depthcat
min_dept
max_dept
bot_dept
temp_sur
tempcat
temp_fs_
lat
long
pos_meth
gear
total
species
number
weight
lat in
common
surtemp
f ishtemp
depth
ispecies
Sorted by:

c yze
t yte

v 4.0 g
':4.0g
3.0g
v 3.0g
* 3.0g
*3.0g
cyze
* 5.0g
* 3.0g
* 2.0g
-9.0g
byte
* 4.0g
* 8 . Og
ty-e
-9.0g
* 8.0g
* 9.0g
d -ble *9.0g
str31
*3 Is
str27
^27s
fl cat
* 9.0g
flcat *9.0g
*9.0g
b-.-ce
h 9.0g

63

Operation of gear
Category of depth
Depth (minumum)
Depth (maximum)
Depth (bottom if MWT)
Temperature (surface)
Category of temperature
Temperature (fishing depth)
Latitude (decimal)
Longitude (decimal)
Gear

Species
Number of individual fish
Catch weight in kilograms
Species — Latin name
Species — common name
Surface temperature degrees C
Fishing depth temperature C
Mean trawl depth in meters
Indicator species

id

describe the data (above), Stata reports “46.09% ofmemory free,” meaning not 46% of the
computer s total resources, but 46% of the 20 megabytes we allocated for Stata data. It is
usually advisable to ask for more memory than our data actually require. Many statistical and
a a-management operations consume additional memory, in part because they temporarily
create new variables as they work.
It is possible to set memory to values higher than the computer’s available physical
memory. In that case, Stata uses “virtual memory,” which is really disk storage. Although
vrrtual memory allows bypassing hardware limitations, it can be terribly slow. If you regularly
wor with datasets that push the limits of your computer, you might soon conclude that it is
time to buy more memory.
Type help limits to see a list of limitations in Stata, not only on dataset size but also
,0S.rJimLnT"SJndJUd,2g matr« siz^command lengths, lengths of names, and numbers of
variables in commands. Some of these limitations
----------- j can be adjusted by the user.

I

I .

I

1
s
i

Graphs

Graphs appear in every chapter of this book —one i - ‘
indication of their value and integration
with other analyses in Stata. Indeed, graphics have a! ways been one of StataTs^g ^nd
reason enough for many users to choose Stata over other packages. The graph Command
evolved incrementally from Stata versions 1 through 7. Stata version 8 marked a major step
forward, howevet. graph underwent a fundamental redesign, expanding its capabilities fo^
sophisticated, publication-quality analvtical graphics. Output appearance and choices were
i tuch imptoved as well. With the new graph command syntax and defaults, or alternatively
C' ne:,meni'S’ attrauCtive (and Publishable) baste graphs are quite easy to draw
phically ambitious users who visualize non-basic graphs will find their efforts supported by
XlyJ.mPreSS1Ve anay °f t0°IS and °PtiOnS‘ deSCribed in the 500-page Graphics Reference
In the much shorter space of this chapter, the spectrum from elementary
j to creative
graphing will be covered taking an cexample’ rather

than syntax-oriented approach (see the
Graphics Reference Manual qv help
? <graph for thorough coverage of syntax). We begin
by illustrating seven basic types of graphs.

histogram

histograms

graph twoway

two-variable scatterplots, line plots, and many others

graph matrix

scatterplot matrices

graph box

box plots

graph pie

pie charts

graph bar

bar charts

graph dot

dot plots

twoway °!vpeSe

°PtiOnS’ Th3t iS esPeciallytrue for the versatile

More specialized graphs such as symmetry plots, quantile plots, and quantile-normal plots
of e anhs fo’ f7Xam;ninS details of variable distributions. A few examples of these, and also
of graphs for industrial quality control, appear in this chapter. Type help graph other
for more details.
Finally, the chapter concludes with techniques particularly useful inbuildingdata-rich selfcontained graph.cs for publication. Such techniques include adding text to graphs, overlayina
nultiple twoway plots, retrieving and reformatting saved graphs, and combining multiple
graphs into one. As our graphing commands grow more complicated, simple batch programs

64

Graphs

65

(do-files) can help to write and re-use them. The full range of graphical choices goes far
beyond what this book can cover, but the concluding examples point out a few of the
possibilities. Later chapters supply further examples.
The Graphics menu provides point-and-click access to most of these graphing procedures.
A note to long-time Stata users: The graphical capabilities of Stata 8 and 9 outshine those
of earlier versions. For analysts comfortable with old Stata, there is much new material to
learn. Menus allow a quick entry, and the new graphics commands, like the old ones, follow
a consistent logic that becomes clear with practice. Fortunately, the changeover need not be
sudden. Version 7-style graphics remain available if needed. They have been moved to the
command graph? . For example, an old-version scatterplot would formerly have been drawn
by the command
graph income education

which does not work in the newer Stata. Instead, the command
.

graph? income education

will reproduce the familiar old type of graph. The options of graph? are similar to those of
the old-style graph . To see an updated version of this same scatterplot, type the new
graphics command
• graph twoway scatter income education

Further examples of new commands appear in the next section, which should give a sense of
what has changed (and what is familiar) with the redesigned graphical capabilities.

Example Commands
histogram y, frequency

Draws histogram of variable y, showing frequencies on the vertical axis.
. histogram y, start(O) width(10) norm fraction

Draws histogram of with bins 10 units wide, starting at 0. Adds a normal curve based on
the sample mean and standard deviation, and shows fraction of the data on the vertical axis.
. histogram y, by(x, total) fraction

In one figure, draws separate histograms of y for each value of x, and also a ’’total”
histogram for the sample as a whole.
kdensity x, generate(xpoints xdensity) width(20) biweight

Produces and graphs kernel density estimate of the distribution of x. Two new variables
are created: xpoints containing thex values at which the density is estimated, and xdensity’
with the density estimates themselves, width (20) specifies the halfwidth of the kernel,
in units of the variable x. (If width () is not specified, the default follows a simple
formula for “optimal.”) The biweight option in this example calls for a biweight
kernel, instead of the default epanechnikov .
I

. graph twoway scatter y x

I

Displays a basic two-variable scatterplot of v against x.

66

Stat/st/cs with Stata

. graph twoway Ifit y x

scatter y x

||

Visualizes the linear regression of y on x by overlaying two twoway graphs: the
regression (linear fit or Ifit ) line, and the y vs. x scatterplot To include a 95%
confidence band for the regression line, replace Ifit with If itci .

I

. graph twoway scatter y x, xlabel(0(10)100) ylabel(-3 (1)6, horizontal)

Constructs scatterplot ofy vs. x, withxaxis labeled atO, 10,..., 100. y axis is labeled at -3,
-2,..., 6, with labels written horizontally instead of vertically (the default).
. graph twoway scatter y x, mlabel(country)

Constructs scatterplot ofy vs. x, with data points (markers) labeled by the values ofvariable
country.
. graph twoway scatter y xl, by(x2)

In one figure, draws separate^ vs. xl scatterplots for each value ofx2.
. graph twoway scatter y xl

[fweight = population], msymbol(Oh)

Draws a scatterplot ofy vs. xl. Marker symbols are hollow circles (Oh), with their size
(area) proportional to frequency-weight variable population.
. graph twoway connected y time

A basic time plot ofy against time. Data points are shown connected by line segments. To
include line segments but no data-point markers, use line instead of connected.:
• graph twoway line y time

. graph twoway line yl y2 time

Draws a time plot (in this example, a line plot) with twoy variables that both have the same
scale, and are graphed against an x variable named time.
. graph twoway line yl time, yaxis(l)

||

line y2 time, yaxis(2)

Draws a time plot with two y variables that have different scales, by overlaying two
individual line plots. The left-handy axis, yaxis (1), gives the scale foryf, while the
right-handy axis, yaxis (2), gives the scale fory2.
. graph matrix xl x2 x3 x4 y

Constructs a scatterplot matrix, showing all possible scatterplot pairs among the variables
listed.
. graph box yl y2 y3

Constructs box plots ofvariablesy7,y2, andyJ.
. graph box y, over(x) yline(.22)

Constructs box plots ofy for each value ofx, and draws a horizontal line aty

.22.

. graph pie a b c, pie

Draws one pie chart with slices indicating the relative amounts of variables a, b, and c. The
variables must have similar units.
.

graph bar

(sum)

a b c

Shows the sums of variables a, b, and c as side-by-side bars in a bar chart. To obtain means
instead of sums, type graph bar (mean) a b c . Other options include bars
representing medians, percentiles, or counts of each variable.
• graph bar

(mean)

a, over(x)

Diaws a bar chart showing the mean of variable a at each value of variable x.

A*.

Graphs



graph bar

(asis)

a b c,

over(x)

67

stack

Draws a bar chart in which the values (“as is”) of variables a, b, and c are stacked on top
of one another, at each value of variable x.


graph dot

(median)

y,

over(x)

Draws a dot plot, in which dots along a horizontal scale mark the median value of v at each
level ofx. Other options include means, percentiles, or counts of each \ ariable.
.

qnorm y

Draws a quantile-normal plot (normal probability plot) showing quantiles of v versus
corresponding quantiles of a normal distribution.
rchart xl x2 x3 x4 x5, connect(1)

Constructs a quality-control R chart graphing the range of values represented by variables

Graph options, such as those controlling titles, labels, and tick marks on the axes are
common across graph types wherever this makes sense. Moreover, the underlying losic of
Stata’s graph commands is consistent from one type to the next. These common elemems are
the key to gaining graph-building fluency,
„, as the basics begin to fall into place.
Histograms
Histograms, displaying the distribution of measurement variables, are most easily produced
with their own command histogram. For examples, we turn to states.dta, which contains
selected environment and education measures on the 50 U.S. states plus the District of
Columbia (data from the League of Conservation Voters 1991; National Center for Education
Statistics 1992, 1993; World Resources Institute 1993).
. use states
(U.S. states data 1990-91)
describe

Contains data from c:\data\states.dta
obs:
si
vars:
21
size:
4,080 (99.9% of memory free)
variable name

I

state
region
Pop
area
density
metro
waste
energy
miles
toxic
green
house
senate
csa t
vsa t

storage
type

display
format

str20
byte
float
float
float
float
flcat
int
in t
float
float
byte
byte

* 9.0g
•9.0g
* 9.0g
*7.2f
*5. If
*5.2f
%8.0g
*8.0g
*5.2f
*5.2f
*8.0g
*8.0g

in:

* 9.0g
*8.0g

value
label

*20s

region

U.S. states data 1990-91
4 Jul 2005 12:07

variable label
State
Geographical regie.-.
1990 population
Land area, square riles

People per square
Metropolitan area emulation, *
Per capita solid waste, tons
Per capita energy s.-.sumed, 5tu
Per capita miles/ye =r, 1,000
Per capita toxics released, les
Per’capita greenhouse gas, ter.s
House '91 environ. voting,
Senate '91 environ. voting, *
Mean composite SAT s c o r e
Mean verbal SAT score

68

Statistics with Stata

msa t
percent
expense
income
high
college

Sorted by:

in t
byte
int
long
float
float

%8.0g
%9 . Og
%9.0g
% 10.0 g
%9.0g
%9.0g

Mean math SAT score
% HS graduates taking SAT
Per pupil expenditures prim&sec
Median household income, $1,000
% adults HS diploma
% adults college degree

I

state

Figure 3.1 shows a simple histogram of college, the percentage of a state’s over-25
population with a bachelor’s degree or higher. It was produced by the following command:
. histogram college, frequency title("Figure 3.1")

Figure 3.1

Figure 3.1

o
CXI

in

-iw;



0)

■W

in

’J.-; •





••

1

o

10

15

20
25
% adults college degree

30

35

Under the Prefs — Graph Preferences menus, we have the choice of several pre-designed
schemes for the default colors and shading of our graphs. Custom schemes can be defined
as well. The examples in this book employ the s2 mono (monochrome) scheme, which among
other things calls for shaded margins around each graph. The s1 mono scheme does not have
such margins. Experimenting with the different monochrome and color schemes helps to
determine which works best for a particular purpose. A graph drawn and saved under one
scheme can subsequently be retrieved and re-saved under a different one, as described later in
this chapter.

Options can be listed in any order following the comma in a graph command. Figure 3.1
i Hush ates two options: frequency (instead ofdensity, the default) is shown on the vertical axis;
and the title Figure 3.1 appears over the graph. Once a graph is onscreen, menu choices
provide the easiest way to print it, save it to disk, or cut and paste it into another program such
as a word processor.
Figure 3.1 reveals the positive skew of this distribution, with ajnode above 15 and an
outlier around 35. It is hard to describe the graph more specifically because the bars do not line
up withx-axis tick marks. Figure 3.2 contains a version with several improvements (based on
some quick experiments to find the right values):

I

Graphs

1.

The ,v axis is labeled from 12 to 34, in increments of 2.

2.

The y axis is labeled from 0 to 12, in increments of 2.

3.

Tick marks are drawn on the r axis from 1 to 13. in increments of 2.

69

4. The histogram’s first bar (bin) starts at 12.

5. The width of each bar (bin) is 2.
histogram college, f
frequency title("Figure 3.2") xlabel(12(2)34)
ylabel(0 (2)12) ytick (1 (2)13)
start(12)
'
) width(2)
Figure 3.2

Figure 3.2

S'-

CM

O

o'00
c

W-

0)

BI

l’<D

uZ

- '

<

n

CM

Bi

o



14

16

A

IW-

Illi i
18o/ 20
22
24
26
% adults college degree

28

30

32

34

Figure 3.2 helps us to describe the distribution more specifically. For example, we now see that
in 13 states, the percent with college degrees is between approximately ir
16 andj m
18.
Other useful histogram options include:

I

i

bin (#)

Draw a histogram with # bins (bars). We can specify either bin (#) or, as
inFigure3.2, start(#) and width (#)—but not both.

percent

Show percentages on the vertical axis, ylabel and ytick then refer to
percentage values. Another possibility, frequency, is illustrated in Figure
3.2. We could also ask for fraction of the data. The default histogram
shows density, meaning that bars are scaled so that the sum of their areas
equals 1.

gap(#)

Leave a gap between bars. # is relative, 0 s # < 100; experiment to find a
suitable value.

addlabels

Label the heights of histogram bars. A separate option, addlabopts ,
controls the how the labels look.

discrete

Specify discrete data, requiring one bar for each value ofx.

70

Statistics with Stata

norm

Overlay a normal curve on the histogram, based on sample mean and standard
deviation.

kdensity

Overlay a kemal-density estimate on the histogram. The option kdenopts
controls density computation; see help kdensity for details.

With histograms or most other graphs, we can also override the defaults and specify our
own titles for the horizontal and vertical axes. The option ytitle controls r-axis titles, and
xtitle controls .v-axis titles. Figure 3.3 illustrates such titles, together with some other
histogram options. Note the incremental buildup from basic (Figure 3.1) to more elaborate
(Figure 3.3) graphs. This is the usual pattern of graph construction in Stata: we start simply,
then experimentally add options to earlier commands retrieved from the Review window, as we
work toward an image that most clearly presents our findings. Figure 3.3 actually is over­
elaborate, but drawn here to show off multiple options.
. histogram college, f
frequency titie("Figure 3.3") ylabel(0(2)12)
ytick(1 (2)13) xlabel
’ (12(2)34)
‘ \
) start(12) width(2) addlabel
norm gap (15)
Figure 3.3

Figure 3.3

13

cxj
O

i

o'00
c
o
6 \

a

LL

iu

4

Bai

CXI

o
12

14

16

18

20
22
24
26
% adults college degree

28

30

32

34

Suppose we w ant to see how the distribution of college varies by region. The by option
obtains a separate histogram tor each value of region. Other options work as they do for single
histograms. Figure 3.4 show s an example in which we ask for percentages on the vertical axis,
and the data grouped into 8 bins.

I

4

Graphs

71

histogram college, by(region) percent bin (8)

West

Figure 3.4

N. East

s8-

s

ao _

s

South

o

CL

- I^r is

Z1

o

Midwest

§ -

8*

1
o _

<

‘r'a-

]

o

10

15

20

25

30

10

15

20

30

25

% adults college degree
Graphs by Geographical region

i

Figure 3.5, below, contains a similar set of four regional graphs, but includes a fifth that
shows the distribution for all regions combined.
. histogram college, percent bin (8) by(region, total)

West

N. East

I

$-

88o

o
o
o

10

Midwest

CL

o _

15



20

25

30

Total

N

88-

I

"I

I

o .
o -

I

■■

8p
test

o _

Figure 3.5

South

10

15

20

a
25

30 10

15

20

25

30

% adults college degree
Graphs by Geographical region
I

Axis labeling, tick marks, titles, and the by (varname) or by (vaniame, total)
options work in a similar fashion with other Stata graphing commands, as seen in the following
sections.

72

Statistics with Stata

Scatterplots
Basic scatterplots are obtained through commands of the general form
graph twoway scatter y x

.

wherey is the vertical or i-a.xis variable, and .v the horizontal or.v-axis one. For example, asain
usingthe states.dta dataset, we could plot waste (per capita solid wastes) against metro (percent
population in metropolitan areas), with the result shown in Figure 3.6. Each point in Figure 3 6
represents one of the 50 U.S. states (or Washington DC).
. graph twoway scatter waste metro
Figure 3.6

9

c
o

e

of

9

w
co
$

I
e

ro
Q.

o
CD

O.

o

ID -

o H—

20.0

40-0 t
60.0
80.0
Metropolitan area population. %

100.0

As with histograms, we can use xlabel, xtick, xti tie. etc. to control axis labels,
tick marks or titles. Scatterplots also allow control of the shape, color, size, and other
attributes of markers. Figure j.6 employs the default markers, which are solid circles The
same effect would result if we included the option msysbol (circle), or wrote this option
in a reviate orm as msymbol (O). msymbol (diamond) or msymbol (D) would
produce a graph with diamond markers, and so forth. The following table lists possible shapes.

msymbol( )

Abbreviation

Description

circle

O

circle, solid

diamond

D

triangle
square
plus
x

T

diamond, solid
triangle, solid

S
+

square, solid
plus sign

X

letter x

smcircle

o

small circle, solid

Graphs

smdiamend
smsquare
smtriangle
smplus
smx

circle__hollow
diamond_hollow
triangle_hollow
square__hollow
smcircle__hollow
smdi amond__ho 11 ow
smtriangle__hollow
sms guare_hol low
point
none

d
s
t
smplus
x
Oh
Dh
Th
Sh
oh
dh
th
sh

P

73

small diamond, solid
small square, solid

small triangle, solid
small plus sign

small letter x
circle, hollow

diamond, hollow



triangle, hollow
square, hollow
small circle, hollow
small diamond, hollow
small triangle, hollow
small square, hollow
very small dot

invisible

The mcolor option controls marker colors. For example, the command
. graph twoway scatter waste metro, msymbol(S) mcolor(purple)

would produce a scatterplot in which the symbols were large purple squares. Type help
colorstyle for a list of available colors.

I

One interesting possibility with scatterplots is to make symbol size (area) proportional to
a third variable thereby giving the data points different visual “weight.” For example we
might redraw the scatterplot of waste against metro, but make the symbols size reflect each
state s population (pop). This can be done as shown in Figure 3.7, using the fweiqht [ ]
(frequency weight) feature. Hollow circles, msymbol (Oh), provide a suitable shape.

Frequency weights are useful with some other graph types as well. Weighting can be a
deceptively complex topic, because “weights” come in several types, and have different
meanings in different contexts. For an overview of weighting in Stata, type help weight.

.—.

unnwm..

9

74

Statistics with Stata

5

■ graph twoway scatter waste metro [fweight = pop],
msymbol(Oh)

Figure 3.7

-

c

o
o'
w

o

A

$

~O -

w
re
o.
re

O

o

o

O
O

Q_

o

o
in
o

20.0

400

60.0

80 0

WO.O

Metropolitan area population, %

$tata density-distribution sunflower plots provide an alternative to scatterplots
i i high-density data. Basically, they resemble scatteiplots in which some of the individual
data pomts are replaced with sunflower-like symbols to indicate more than one observation at
that location. Figure 3.8 shows a sunflower-plot version of Figure 3.6, in which some of the
flower symbols (those with four “petals”) represent up to four individual data points, or states
table printed after the sunflower command provides a key regarding how many
o servations each flower represents. The number of petals and the darkness of the flower
correspond to the density of data.
sunflower waste metro, addplot(Ifit waste
metro)
Bin width
11.3714
Bin height
.286522
Bin aspect ratio
. 0218209
Max obs in a bin
4
Light
3
Dark
13
X-center
67.55
Y-center
. 96
Petal weight
1

flower
type

petal
weight

No. of
petals

No . of
flowers

estimated
obs.

actual
obs .

none
light
light

1
1

3
-4

5
3

23
15
12

23
15
12

50

50

Graphs

75

Figure 3.8

(D O
"TO v-’

o

o

o

■o

o
o

<D
U.



7^

I



o

o

<D o
to o
ro

o

o-4-

$

TJ

i
i

o

.

——

o

Ks?"

o

ro
Q.

ro



hO

o

i

o

I—

20.0

I



o

40.0
6Q o
go o
_________ Metropolitan area population, %'

100.0

o Per capita solid waste, tons
1 petal = 1 obs
» 1 petal = 1 obs.
----------Fitted values

.< Si2T£S“iCH “y h"S5‘1 Wi,h larEe
™»y observations plot
“ ""I (" ;den,1“l) «»«l™tes. The example in Figure 3.8 inclodes a regression line

’dd'd “

the graph into a visual jumble. Concentrating on one region such as the West seems more
fiZXTpX Xf qUaIlfieraCCOmplisheS this> Producing Results seen in Figure 3.9 on the

I

3

76

Statistics with Stata

• graph twoway scatter waste metro if region==l, mlabel(state)

s-

Figure 3.9
• California

o
XT

-

C T-

o
(D

-

• Oregdft Hawai*

■g

• Washington

So _

• New Mexico

Q-X-

O

• Alaska

Oo

c Idaho

CO O

• Nevada
• Arizona

• Montana
• Wyoming

iV
20.0

• Colorado
• Utah

40 0
~
BOX)
'
8CL0
Metropolitan area population, %

100.0

Figure 3.10 (below) shows separate waste - metro scatterplots for each region. The
relationship between these two variables appears noticeably steeper in the South and Midwest
than it does in the West and Northeast, an impression we will later confirm. The ylabel and
xlabel options in this example give they- and x-axis labels three-digit (maximum) fixed
display formats with no decimals, making them easier to read in the small subplots.
. graph twoway scatter waste metro, by(region)
ylabel(, format(%3.0f)) xlabel(, format(%3.Of))
Figure 3.10

West
CXI

c
O

<D
W
CQ

£ -H

•g

South

° CM -I
ro

Midwest

Q.
O

O

Q.

r- c

20

4C?

60

80

100 20

40

Metropolitan area population, %

Graphs by Geographical region

60

80

100

Graphs

77

Scatterplot matrices, produced by graph matrix. prove useful in multivariate analysis.
They provide a compact display of the relationships between a number of variable pairs,
allowing the analyst to scan for signs of nonlinearity, outliers, or clustering that mieht affect
statistical modeling. Figure 3.11 shows a scatterplot matrix involving three variables from
states.dta.
• graph matrix miles metro income waste, half msymbol(oh)

Figure 3.11

Per
capita
miles/year,
1,000
100.0- 0

O

o
0° flp

s "^o?> o
50.0-

Metropolitan
area
population,

0

OoO® 0

c

’•« o
o oo

%8:

%

OoO

o

o

Median
household
income,
$1,000

40- 0
-

0

o

30c o°

iJ8=-

0 0°

o°0

0
°0 e

0

■>

0^0

O

1.00-

oc<f><3° ’
O

O

o
Ooo
0%°c

o
cP

c0

'''

*QO

O.

0

°

Per
capita
solid
waste,
tons

OO

. o

0.50
6

8

10

12 0.0

50.0

10(20

30

40

50

The half option specified that Figure 3.11 should includeonly the lower triangular part
of the matrix. The upper triangular part is symmetrical and. for many purposes, redundant,
msymbol (oh) called for small hollow circles as markers, just as we might with a scatterplot.
Control of the axes is more complicated, because there are as many axes as variables- type
help graph_matrix for details.
When the variables of interest include one dependent or "effect’' variable, and several
independent or cause” variables, it helps to list the dependent variable last in the graph
matrix variable list. That results in a neat row of dependent-versus-independent variable
graphs across the bottom

Line Plots

I
I

Mechanically, one
line piots
plots are scatterplots m
in which the points are connected by line segments.
iv.^uaiHcany,
Like scatterplots, the various types of line plots belong to Stata’s versatile graph twoway
family. The scatterplot options that control axis labelingand markers work much the same with
line plots, too. New options control the characteristics of the lines themselves.
Line plots tend to have different uses than scatterplots. For example, as time plots they
depict changes in a variable over time. Dataset cod.dta contains time-series data reflecting the

I

Statistics with Stata

78
4

unhappy story of Newfoundland s Northern Cod fishery. This fishery, which had been among
the world s richest, collapsed in 1992 primarily due to overfishing.
Contains data from C:\data\cod.dta
obs :
38
vars :
size :

variable name

storage
type

year
cod
Canada
TAG
biomass
Sorted by:

Newfoundland's Northern Cod
fishery, 1960-1997
4 Jul 2005 15:02

5
684 (99.9% of memory free)

int
float
int
int
float

display
format

value
label

variable label

%8.0g
%8 . Og
%8.0g
%8.0g
%9.0g

Year

Total landings, lOOOt
Canadian landings, lOOOt
Total Allowable Catch, lOOOt
Estimated biomass, lOOOt

year

A simple time plot showing Canadian and total landingscan be constructed by drawing line
graphs of both variables against year. Figure 3.12 does this, showing the “killer spike” of
international overfishing in the late 1960s, followed by a decade of Canadian fishing pressure
in the 1980s, leading up to the 1992 collapse of the Northern Cod.

. graph twoway line cod Canada year
Figure 3.12
V)
O)

c



c
ro
■Q
CD

Sg O

o
o

-

\\
x\
\\

.TO

ro
oO
H

-L^
I960

1970

1980
Year

1990

2000

Total landings, 1000t __ -- Canadian landings, 10OOt |

In Figure 3.12, Stata automatically chose a solid line for the first-named^ variable, cod, and
a dashed line for the second, Canada. A legend at the bottom explains these meanings. We
could improve this graph by rearranging the legend, and suppressing the redundant j,-axis title,
as illustrated in Figure 3.13.

Graphs



graph twoway line cod Canada year,

legend(label

79

(1 "all nation®-^

ring (0)
Figure 3.13

-------- - all nations!
---------- Canada

I
\\
\\
\\
\\
\\

v

o -

1960

1970

1980
Year

1990

2000

I

The legend option for Figure 3.13 breaks down
as follows. Note that all of these
suboptions occur within the parentheses followine legend

I

label (1

"allnations")

label first-namedvariable “all nations”

label (2

"Canada")

label second-named,v variable “Canada”

position (2)

piace the Iegend at 2 o,clock position (upper right)

ring (01

Place the legend within the plot space

rOWS (2 J

organize the legend to have two rows

toyshowrttehneidSatthe
‘abelS
Pl3CinS tHem Uithin tfK pl0t SpaCe- we Ieave more room
to show the data and create a more attractive, readable figure, legend works similarlv for
other graph styles that have legends. Type help legend_option to see a list ofthe man v
suboptions available.

- - I‘g.U_r!S2‘!2 and 3-13 s’mPly connect each data point with line segments. Several other
connecting styles are possible, using the? connect option. For example.

connect(stairstep)
or equivalently.
connect(J)
will cause points to be connected in stairstep (flat, then vertical) fashion. Figure 3.14 illustrates
wi h a stairstep time plot of the government-set Total Allowable Catch (TAG) variable from
v t/ Lc . C< < C< ,

80

Statistics with Stata

• graph twoway line TAC year.

connect(stairstep)

o
o

Figure 3.14

00

oo
oo
O (D
£

28-

X> XT

ro
jp

<

fSI
1960

1970

1980
Year

1990

2000

Other connect choices are listed below,
The default, straight line segments,
corresponds to connect (direct)
or connect (1). For more details, see help
connectstyle.

connect( )

Abbreviation

_ Description

none

i

do not connect

direct

1 (letter “el”)

connect with straight lines

ascending

L

direct, but only ifx[/+l] > x[z]

stairstep

J

flat, then vertical

stepstair

vertical, then flat

3J 5 Jon
Rowing page) repeats this stairstep plot
of TAC^ but with some
enhancements of axis labels and titles, The option xtitle(""]
t
) requests
no x-axis title
(because “year” is obvious). We added
' J tick marks at two-year intervals to thex axis. labeled
they axis at intervals of 100, and printedly-axis labels horizontally instead of vertically (the
default).

r

Graphs

81

. graph twoway line TAC
year, connect(stairstep)
xtitle(" " )
xtick(1960(2)2000)
ytitle("Thousands of tons")
ylabel(0(100)800, angle(horizontal)) xtitle(
)
cipattern(dash)

800-

Figure 3.15

700"-i
i
l
lI
I
l
i
i
i
I
I

600w

o 500■o 400CTJ
W

J 300-

i
i
i
I
r
“I I

200-

i

I
I
-I
I
I
I

100-

0-|

1960

1970

1980

1990

2000

J^ead of letting Stata determine the line patterns (solid, dashed, etc.) in Figure 3.15 we
used the cipattern (dash) option to call for a dashed line. Possible line pattern choices
are listed in the table below (also see help linepatternstyle ).

I

cipattern( )

Description

solid

solid line

dash

dashed line

dot

dotted line

dash dot

dash then dot

shortdash

short dash

shortdash dot

short dash followed by dot

longdash

long dash

longdash_dot

long dash followed by dot

blank

invisible line

formula

for example, cipattern (-.) or cipattern (—..)

82

Statistics with Stata

Before we move on to other examples and types, Figure 3.16 unites the three variables
discussed in this section to create a single graphic showing the tragedy of the Northern Cod.
Note how the connect ( ). cipattern ( ), and legend ( ) options work in this
three-vanable context.

Ji
!

• graph twoway line cod Canada TAC year, connect(line line stairstep)
cipattern((solid
- - longdash
dash) xtitle("”) xtick(1960 (2)2000)
ytitle("Thousands
— of
_ tons"
---- ') ylabel (0 (100) 800, angle(horizontal))
xtitie("" ) legend(label (1 "all nations") label (2 "Canada")
label (3 "TAC") position(2) ring(0) rows(3))
5

Figure 3.16

800-I

------ all nations
------ Canada
------ TAG

700 J
600-

i

c

2 500-

400^
V)
3
O
XL

300 -i

200
100

°H
1960

1970

1980

1990

2000

Graphs

83

Connected-Line Plots

woway line both apply to graph twoway connected as well. Figure 3.17 shows
a etault example, a connected-line time plot of the cod biomass variable (bio) from cod.dta.
graph twoway connected bio year

.

Figure 3.17

o
io
CXI

o

§s

o
T—

V) O

ro io
o

is
ro o
co

LU
o
m

o

1960

1970

1980
Year

1990

2000

I

The dataset contains only biomass values for 1978 through 1997, resulting in much empty
space in Figure 3.17. if qualifiers allow us to restrict the range of years. Figure 3.1 sXn
e o owing page, does this. It also dresses up the image to show control of marker symbols
me patterns, axes, and legends. With cod landings and biomass both in the same ima-e we
recognJzed. 11355 830 kS
’ 98°S’
before a Crisis was off’cially

I

84

i

Statistics with Stata

• graph twoway connected bio cod year if year > 1977 & year < 1999,
msymbol(T Oh) cipattern(dash solid) xlabel(1978(2)1996)
xtick (197 9 (2) 1997) ytitle ( "Thousands of tons") xtitle("’’ )
ylabel (0 (500) 2500, angle(horizontal))
legend(label (1 "Estimated biomass") label(2 "Total landings")
position(2) rows(2) ring(0))
Figure 3.18
2500-

a--- Estimated biomass

I'

2000-

I
I
I
I
I
I
I

c
o

? 1500-

I



\

I

ID

------ Total landings
I
I
I
I
I
I
I
I
I
I

o 1000- / /

A
/ \
'

\

\
\
\
\
\
\
\

5004
OH,

4

1978 1980 1982 1984 1986 1988 1990 1992 1994 1996

Other Twoway Plot Types
In addition to basic line plots and scatterplots, the graph twoway command encompasses
a wide variety’ of other types. The following table lists the possibilities.

I

graph twoway

Description_____________________________________________

4 ■

scatter

scatterplot

i

line

line plot

connected

connected-line plot

scatteri

scatter with immediate arguments (data given in the command line)

area

line plot with shading

bar

twoway bar plot (different from graph bar)

spike

twoway'spike plot

dropline

dropline plot (spikes dropped vertically or horizontally to given value)

dot

twoway dot plot (different from graph dot)

rarea

range plot, shading the area between high and low values

Is I

Graphs

I

85

rbar

range plot with bars between high and low values

rspike

range plot with spikes between high and low values

reap

range plot with capped spikes

rcapsym

range plot with spikes capped with symbols

rscatter

range plot with scatterplot marker symbols

rline

range plot with lines

rconnected

range plot with lines and markers

pcspike

paired-coordinate plot with spikes

pccapsym

paired-coordinate plot with spikes capped with symbols

pcarrow

paired-coordinate plot with arrows

pebarrow

paired-coordinate plot with arrows having two heads

pcscatter

paired-coordinate plot with markers

pci

pcspike with immediate arguments

pcarrowi

pcarrow with immediate arguments

tsline

time-series plot

tsrline

time-series range plot

mband

straight line segments connect the (x, y) cross-medians within bands

mspline

cubic spline curve connects the (x, v) cross-medians within bands

lowess

LOWESS (locally weighted scatterplot smoothing) curve

Ifit

linear regression line

qfit

quadratic regression curve

fpfit

fractional polynomial plot

Ifitci

linear regression line with confidence band

qfitci

quadratic regression curve with confidence band

fpfitci

fractional polynomial plot with confidence band

function

line plot of function

histogram

histogram plot

kdensity

kernel density plot

The usual options to control line patterns, marker symbols, and so forth work where
appropriate with all twoway commands. For more information about a particular command,
type help twoway_mband, help twoway__function, etc. (using any of the names
above). Note that graph twoway bar is a different command from graph bar .
Similarly, graph twoway dot differs from graph dot . The twoway versions
JI

86

Statistics with Stata

provide various methods for plotting a measurement y variable against a measurement _v
variable, analogous to a scatterplot or a line plot. The non-twoway versions, on the other hand,
provide ways to plot summary statistics (such as means or medians) of one or more
measurementy variables against categories of one or more* variables. The twoway versions
thus are comparatively specialized, although (as with all twoway plots) they can be overlaid
with other twoway plots for more complex graphical effects.
Many of these plot types are most useful in composite figures, constructed by overlaying
two or more simple plots as described later in this chapter. Others produce nice stand-alone
graphs. For example, Figure 3.19 shows an area plot of the Newfoundland cod landings.


graph twoway area cod Canada year, ytitle("")

Figure 3.19

O

o
co

o
o

<D

O
O

O
O
CN

o
1960

1970
Total landings, 10001

1980
Year

1990

2000

Canadian landings, 1000t

The shading in area graphs and other types with shaded regions can be controlled throush
the option bcolor . Type help colorstyle for a list of the available colors, which
include gray scales. The darkest gray, gsO, is actually black. The lightest gray, gsl6, is white.
Other values are in between. For example, Figure 3.20 shows alight-gray version of this graph.

Graphs

87

5raph twoway area cod Canada year, ytitle ('”•
) bcolor(gsl2 gsl4)
o
o
oo

Figure 3.20

o
o
co

o
o

I

o
o

CXI

o

1960

1970

1980
Year

1990

Total landings, 1000t

2000

Canadian landings, 1000t

Unusually cold atmosphere/ocean conditions played a secondary role in Newfoundland’s
fisheries disaster, which involved not only the Northern Cod but also other species and
populations. For example, key fish species in the neighboring Gulf of St. Lawrence declined
during this period as well (Hamilton, Haedrich and Duncan 2003). Dataset gulfdta describes
environment and Northern Gulf cod catches (raw data from DFO 2003).
Contains data from C:\data\gulf.dta
obs:
56

vars :
size:

7
1,344 (99.9% of memory free)

storage
type

display
format

winter
minarea
maxarea
mindays
maxdays
cil

int
float
float
byte
byte
float

%8.0g
%9. Og
%9.0g
%8. Og
%8.0g
%9. Og

cod

float

%9. Og

variable name

Sorted by:

value
label

Gulf of St. awrence
envircnme
and cod fishery
10 Jul 2005 11:51

variable label

Winter
Minimum ice area, 1090 km ■■'2
Maximum ice area. 1000 km''2
Minimum ice days
Maximum ice days
Cold Intermediate Layer
temperature mini mum, C
N. Gulf cod catch. 1000 tons

winter

The maximum annual ice cover averaged 173,017 km2 during these years.
summarize maxarea

1

Variable

|

Obs

Mean

Std. Dev.

Min

Max

maxarea

|

38

173.0172

37.18623

47.8901

220.1905

88

7

Statistics with Stata

Figure 3.21 uses this mean (173 thousand) as the base for a spike plot, in which spikes
above and below the line show above and below-average ice cover, respectively. The
yline (173) option draws a horizontal line at 173.
. graph twoway spike maxarea winter if winter > 1963, base(173)
yline(173) ylabel(40(20)220, angle(horizontal))
xlabel (19 65 (5) 2000)
Figure 3.21

220-

200CN

<E 1801

I

l

I

T7

g 160-

o

rn

co 140O)

co

g 1203 100-

E

X

rc

80-

6040-|
1965

1970

1975

1980

1985

Winter

WO

1995

2000

The base () format of Figure 3.21 emphasizes the succession of unusually harsh winters
(above-average maximum ice cover) during the late 1980s and early 1990s, around the time of
Newfoundland’s fisheries crisis. We also see an earlier spell of mild winters in the early 1980s,
and hints of a recent warming trend.
A different view of the same data, in Figure 3.22, employs lowess regression to smooth the
time series. The bandwidth option, bwidth (. 4), specifies a curve based on smoothed data
points that are calculated from weighted regressions within a moving band containing 40% of
the sample. Lower bandwidths such as bwidth (. 2), or 20% of the data, would give us a
more jagged, less smoothed curve that more closely resembles the raw data. Higher bandwidths
such as bwidth (. 8), the default, will smooth more radically. Regardless of the bandwidth
chosen, smoothed points towards either extreme of the x values must be calculated from
increasingly narrow bands, and therefore will show less smoothing. Chapter 8 contains more
about lowess smoothing.

Graphs

89

graph twoway lowess maxarea winter if winter > 1963, bwidth(.4)
yline(173) ylabel(40(20)220, angle(horizontal))
xlabel(1965 (5) 2000)



Figure 3.22

220-1

200180(D

.? 160$
CD

2 140x
E 120w
(D 100_O

80-

6040__

1965

1970

1975

1980
1985
Winter

1990

1995

2000

Range plots connect high and low j- values at each level ofx, using bars, spikes, or shaded
areas. Daily stock market prices are often graphed in this way. Figure 3.23 shows a capped­
spike range plot using the minimum and maximum ice cover variables from gulfdta.
graph twoway reap minarea maxarea winter if winter > 1963,
ylabel(0 (20)220, angle (horizontal)) ytitle("Ice area, 1000 kinA2")
xlabel(1965 (5)2000)

-

Figure 3.23

220-j

-I I I

200-J

1804

>

T

T-t

I1

I

■-

. 1601l

CM

E 140 J

| 12°1

TI

ra- IDOcu
o 80-I
o
600)

I

40-

I

20-

o-l
1965

-L___

1970

1975

1980
1985
Winter

1990

1995

2000

90

Statistics with Stata

These examples by no means exhaust the possibilities for twoway graphs. Other
applications appear throughout the book. Later in this chapter, we will see examples involving
overlays of two or more twoway graphs, forming a single image.

Box Plots
Box plots convey information about center, spread, symmetry, and outliers at a glance. To
obtain a single box plot, type a command of the form
.

graph box y

If several different variables have roughly similar scales, we can visually compare their
distributions through commands of the form


graph box

w x y z

One of the most common applications for box plots involves comparine the distribution of
one variable over categories of a second. Figure 3.24 compares the distribution of college
across states of four U.S. regions, from dataset states.dta.
graph box college,



yline(19.1)

over(region)

Figure 3.24

o
co ■

io
0) CXI
<D

<D
•u
0)
O)
=O
©cq
w

I

3

TJ
(U
o> in

i
West

N. East

South

Midwest

e median proportion of adults with college degrees tends to be highest in the Northeast
and lowest in the South. On the other hand, southern states are more variable. Regional
medians (lines within boxes) in Figure 3.24 can be compared visually to the 50-state median
in icated by the yline (19.1) option.. This median was obtained by typing
.

summarize college if region < ., detail

Graphs

91

Chapter 4 describes the summarize, detail command. The if region <
qualifier above restricted our analysis to observations that have nonmissine values of regionthat is, to every place except Washington DC.

* ’
The box in a box plot extends from approximate first to third quartiles, a distance called the
interquartile range (IQR). It therefore contains approximately the middle 50% of the data
Outliers, defined as observations more than 1.5IQR beyond the first orthird quartile, are plotted
individually in a box plot. No outliers appear among the four distributions in Figure 3.24.
Stata s box plots define quartiles in the same manner as summarize, detail. Thisisnot
the same approximation used to calculate “fourths” for letter-value displays. Iv (Chapter 4)
See Frigge, Hoaglin, and Iglewicz (1989) and Hamilton (1992b) for more about quartile
approximations and their role in identifying outliers.
Numerous options control the appearance, shading and details of boxes in a box plot see
elp graph_box fora list. Figure 3.25 demonstrates some of these options, and also the
horizontal arrangement of graph hbox . using per capita energy consumption from
states.dta. The option over (region, sort(l) ) calls for boxes sorted inascendina order
according to their medians on the first-named (and in this case, the only) y variable
intensify (30) controls the intensity of shading in the boxes, setting this somewhat lower
(less dark) than the default seen in Figure 3.24. Counterintuitively, the vertical line marking
the overall median (320) in Figure 3.25 requires a yline option, rather than xline .


I

I

graph hbox energy, over(region,

N. East

o

Midwest

d

sort(l)) yline(320) intensity(30)

Figure 3.25

West:

South

I
200

400
600
800
Per capita energy consumed, Btu

1,000

The energy box plots in Figure 3.25 make clear not only the differences among medians,
but also the presence outliers — four very high-consumption states in the West and South
With a bit of further investigation, we find that these are oil-producing states: Wyoming,
Alaska, Texas, and Louisiana. Box plots excel at drawing attention to outliers, which are easily
ovenooked (and often cause trouble) in other steps of statistical analysis.
4

92

Statistics with Stata

Pie Charts
^ charts are popular tools for "presentation graphics,” although they have little value for
analyitcal work. Stata's basic pie chan command has the fomt
.

graph pie w x y z,

pie

where the variables iv, x, y. and all measure
quantities of something in similar units (for
example, all are in dollars, hours, or people).
Dataset AKethnic.dta. on the ethnic composition of Alaska’s population provides an
illustration Alaska’s indigenous Native population divides into three broad cultural/liimuistic

LT (,n,ClUding Athabaska’ Tlingit’ alld Haida)’ and Eskimo (Yupk and
h unTatt TeUt’
nupiat). The variables aleut. mdian. eskimo, and nonnativ are population counts for each
group, taken from the 1990 U.S. Census. This dataset contains onlv three observations
representing three types or sizes of communities: <cities
‘ ’ of------10,000 people or more; towns of
1.000 to 10,000; and villages with fewer than 1.000 people.
Contains data from C:\data\Ar-e
obs :

vars:
_yze:

variable name

comtype
pop
n
aleut
Indian
es kimo
nonnativ

3

7
63 <99-9%

storage
type
byte
float
int
in t
in t
in t
float

dta
me—ry free)

display
format

value
label

% 8.0 g
*9.0g
*8.0g
%8.0g
%8. Cg
%8.0g
%9.0g

popcat

Alaska ethnicity 1990
4 Jul 2005 12:06

variable label

Community type (size)
Population
number of communities
Aleut
Indian
Eskimo
Non-Native

Sorted by:

yhejnajority
’ *
tu
• of the state'sS population
P°PulatI0n is
is non-I
non-Native,
as clearly seen in a pie chart (Figure
explode) causes the third-named variable, eskimo.
Xolof pIT (T
fc shaded a^t

poss.bl.ties such as color(blue)
color (blue) or color (cranberry) exist
Type help
tTb
T'11?5'- Plabel(3 P—nt, gap (20)) causes a percentage label
centr W
T ^"°
sVlcea gaP of 20 relative radial units from the
r X e see that about 8 /« of Alaska’s population is Eskimo (Inupiat or Yupik) The
legend option calls for a four-row box placed at the 11 o’clock position within the plot

a*

4

Graphs

93

• graph pie aleut Indian eskimo nonnativ, pie(3, explode)

Pie (4, color(gsl3)) plabel(3 percent, gap(20))
legend(position(11) rows(4) ring(0))
Aleut
Indian
Eskimo
Non-Native

Figure 3.26
8.072%

Non-Natives are the dominant group in Figure 3.26, but if we draw separate pies for each
type of community by adding a by (comtype) option, new details emerge (Figure 3.27, next
page). The option angleO () specifies the angle of the first slice of pie. Setting this firstslice angle at 0 (horizontal) orients the pies in Figure 3.27 in such a way that the labels are more
readable. The figure shows that whereas Natives are only a small fraction of the population in
Alaska cities, they constitute the majority among those living in villages. In particular, Eskimos
make up a large fraction of villagers — 35% across all villages, and more than 90% in some.
This gives Alaska villages a different character from Alaska cities.

i

94

I

Statfst/cs with Stata

graph pie aleut Indian <eskimo
’'
nonnativ, pie (3, explode)
pie(4, color(gsl3)) plabel(3 percent, gap(8))
legend(rows(1)) by(comtype) angleO(0)

.

villages

Figure 3.27

towns

V

34.67%

8.141%

cities

2332%

;

Indian

Aleut

Eskimo

Non-Native

Graphs by Community type (size)

Bar Charts
though they contain less information than box plots, bar charts provide simple and versatile
displ ays for comparing sets of summary statistics such as means, medians, sums, or counts. To
obtain vertical bars showing the mean ofj> across categories ofx, for example, type
. graph bar (mean) y, over(x)

For horizontal bars showing the sum of across categories ofx7 within categories ofx2, type
- graph hbar

(sum) y, over(xl)

over(x2)

The bar chart could display any of the following statistics:
mean
sd
sum

rawsum
count

max

min

median

Pl

Means (the default; used if the type of statistic is not specified)
Standard deviations
Sums

Sums ignoring optionally specified weight
Numbers of nonmissing observations
Maximums
Minimums
Medians
1st percentiles

Graphs

95

P2

2nd percentiles (and so forth to P9 9 )
Interquartile ranges
Saoter and'afC ™ary Jatis!ics is the same as that
collapse command (see
and XS
t
nUmberofothercommand s including graph dot (next section)
ana table (Chapter 4).
iqr

meaXXm^helQQi? C°nta^
°n the U'S-StateS’ combiniagsocioeconomic
90 ^nsus with several health-risk indicators from the Centers for Disease
Control (2003), averaged over 1994-98.
Contains data from C:\data\statehealth.

obs
~ ‘ :'
vars :
size :

51

1?

3,315 (99.9% of memory free)

variable name

state
region
income
income?
high
college
overweight
inactive
smokeM
smokeF
smokeT
motor
Sorted by:

dta

storage
type

display
format

str20 %20s
byte
%9.0g
long
%10 . Og
float %11.Og
float %9.0g
float %9.0g
float % 9. Og
float %9.0g
float %9. Og
float %9.0g
float %9. Og
float %9. Og

value
label

region

income?

Health indicators 1994-96
9 Jul ?005 11:56

( CDC)

variable label

US State
Geographical region
Median household income, 1990
Median income low cor high
% adults HS diploma,, 1990
% adults college degree, 1990
% overweight
% inactive in leisure time
% male adults smoking
% female adults smoking
% adults smoking
Age-adjusted motor-vehicle
related deaths/100,000

state

es are highest m the South (36%), and lowest in the West (21 %). Note that the vertical axis

X

I

1
-J

eX"”"0, VariaM'

,here iS Only ■>”) ShO"M be f“"‘'

p

96

Statistics with Stata

graph bar (median) inactive, over(region) blabel(bar)
bar(l, bcolor(gslO))
o

Figure 3.28
36.05

o
co

29.1

28.3

>
20.9
^Zo
o oj
o
io
Q.

o

ii

SI

ii



KA

SB

v i;?.

b?W^'v'
o

_______ _
West

2L_____
N. East

South

Midwest

and carina?. (K ll0W7 rC) eIabOrateS thiS idea by addin8 a second variable, ove^eight,

and coloring its bars a darker gray. The bar labels are size (medium) in Figure 3 29
making them larger than the defaults, size (small), used in Figure 3.28. Other possibilities
for size ( ) suboptions include labels that are tiny , medsmall, medlarge or
arge. See help textsizestyle for a complete list. Figure 3.29 shows that regional
differences in he prevalence of overweight individuals are less pronounced than differences
in inactivity, although both variables’ medians are highest in the South and Midwest

Graphs



97

graph bar (median) inactive overweight, over(region)
blabel(bar, size(medium))
bar(l, bcolor (gslO) ) bar(2, bcolor (gs7) )

.

o

Figure 3.29

36.05
31.3

O
co ■

27.6

28.3

ip

27.1

20.9
o

1 I

CM

o

to

j

_ fL

• II

o

West
[Iv--'

I

I

1

Illi

-J



N. East
P 50 of inactive

31.2

29.1

South

I
-

Midwest

P 50 of overweight |

th
t
!
,hl8^lnCOmc states (states having median household incomes below or
above the national median), revealing a striking correlation with wealth. Within each reeion
f^r^'T0"16 h-3!65 eXhlb,t h'gher mean fataIity rates' Across both income cateaories’
tahty rates are higher m the South, and lower in the Northeast. The order of the two 'over
options in the command controls their order in organizing the chart. For this example we chose
nzontal bar chart or hbar . In such horizontal charts, ytitle . yline , etc refer to
the horizontal axis, yline (17.2) marks the overall mean.

*!

98

Statistics with Stata

. graph hbar (mean) motor, over(income2) over(region) yline(17.2)
ytitle("Mean motor-vehicle related fatalities/100,000")
Figure 3.30

Low income I
West

i
High income |

_ _______

Low income

N. East
High income

South

Low income

High income

Low income

Midwest
High income
0

.. 5
10
15
20
25
Mean motor-vehicle related fatalities/100,000

Bars also can be stacked, as shown in Figure 3.31. This plot, based on the Alaska ethnicity
data (AKethmc.dta), employs all the defaults to display ethnic composition by type of
community (village, town, or city).
• graph bar

(sum)

nonnativ aleut Indian eskimo,
over(comtyp)

o
o
o

stack

Figure 3.31

o
o


R

'■

tew®

o
o

UBI

§
o
o
o
• ■ HlHI
O

villages

towns
sum of nonnativ
sum of Indian

cities
sum of aleut
sum of eskimo

Graphs

99

inel ‘gUrC J J-5edrawsthlsPlotwlthbe,terlegendand axis labels. The over option now
The leoenlo0 f
elaFel t,lecommunity
50‘he horizontal axis is more informative
Placedin the |0)Ptl°niSp"ClfieS fourrows in the same ^ical orderas the bars themselves, and
vtitle vl H t
POSIt'On lnSide IhC P'Ot SpaCC- 11 als0 imProves le^nd labels
y tie , ylabel, and ytitle options format the vertical axis.
. graph bar

(sum) nonnativ aleut Indian eskimo

legend(rows(4) order(4 321) position(ll)
ring(0)
label (1 "Non-native") label(2 "Aleut”)
label (3 "Indian") label (4 "Eskimo”))
stack ytitle(Population)
ylabel (0 (100000) 300000) ytick(500 0 0 (100000) 350000)

o
o

§

Figure 3.32

HBBI Eskimo
.'ix'ii'CiSKis Indian
Aleut
Non-native

co

o
3 CN
Q.

Q.

o
o
o

o

Villages <1,000



Towns 1,000-10,000

Cities >10,000

each community type, this bar chart shows their absolute sizes. Consequently Fifure3 32 tells

p:zi'h;^Fx3s27“”ld ,he “jon,y of A“a's
Dot Plots

I

Dot plots serve much the same purpose as bar charts: visually comparing statistical summaries
o one or more measurement variables. The org.mz.tieo i st.’ „pC„Xte ™ «
conma'
rh
'““"S ,l,e
comparing the medians of variables x, y, z, and w, type


graph dot

(median)

x y z w

To See .

P

100

Statistics with Stata

!*!!

For a dot plot comparing the mean of v across categories of.v. type


graph dot

(mean) y, over(x)

Figure 3.33 shows a dot plot of male and female smoking
J rates by region, from
statehealth.dta. The over <option
’ includes
* ‘ a suboption, sort (smokeM ), which calls for
the regions to be sorted in order of their mean values of smokeM

hollow circle for smokeF.
. graph dot (mean) smokeM smokeF, over(region
overfregion,, sort(smokeM)
sort(smokeM)))
marker(1, msymbol(T)) marker(2, msymbol(Oh))

Figure 3.33

West

A

N. East

Midwest

South

0

10

20

▲ mean of smokeM

mean of smokeF

30

Although Figure 3 33 displays only eight means, it does so in a way that facilitates several

of dot plots is their compactness. Dot plots (particularly when rows are sorted by the statistic
of interest, as in Figure 3.33) remain easily readable even with a dozen or more rows.

Symmetry and Quantile Plots

summary graphs, but convey more detailed information.

I

Graphs

101

A histogram of per-capita energy consumption in the 50 U.S. states (from states dta}
PPears m F.gure 3.o4. The distribution includes a handful of very highconsumption states
ich happen to be oil producers. A superimposed normal (Gaussian) curve indicates that
of p^itive skewter t lan'n°mial left tai1’and a heavier-than-normal right tail — the definition
. histogram energy f start(lOO) width(lOO) xlabel(0(100)1000)
frequency norm

LO

Figure 3.34

CM

8

IB’
>

£
5-

E
iiz o

-J.'-’

iS

z

B
i

o

0

100

200

1

wl

300 400
500
600
700
Per capita energy consumed. Btu

800

900

1000

I

i

v X

A

- (O'V

10032

A4-' /
Vfrr^

"

I

>

1^/

> o^/

102

Statistics with Stata

Figure 3.35 depicts this distribution as a symmetry plot. It plots the distance of the zth
observation above the median (vertical) against the distance of the zth observation below the
median. All points would lie on the diagonal line if this distribution were symmetrical. Instead,
we see that distances above the median grow steadily larger than corresponding distances below
the median, a symptom of positive skew. Unlike Figure 3.34, Figure 3.35 also reveals that the
energy-consumption distribution is approximately symmetrical near its center.
symplot energy
Figure 3.35

Per capita energy consumed, Btu ■

o
co

§ <O
<D

E
0)

-I
8
I
w
CM

o



0

50
100
Distance below median

150

Quantiles are values below which a certain fraction of the data lie. For example, a .3
quantile is that value higher than 30% of the data. If we sort n observations in ascending order,
the zth value forms the (z - .5)/zz quantile. The following commands would calculate quantiles
of variable energy:
. drop if energy >=
sort energy
. generate quant =

(_n -

.5)/ N

As mentioned in Chapter 2, _n and _N are Stata system variables, always unobtrusively present
when there are data in memory'. _n represents the current observation number, and _N the total
number of observations.

"1
Au.

Graphs

103

Quantile plots automatically calculate what fraction of the observations lie below each data
value, and display the results graphically as in Figure 3.36. Quantile plots provide a graphic
reference for someone who does not have the original data at hand. From well-labeled quantile
plots, we can estimate order statistics such as median (.5 quantile) or quartiles (.25 and .75
quantiles). The IQR equals the rise between .25 and .75 quantiles. We could also read a
quantile plot to estimate the fraction of observations falling below a given value.
quantile energy
o
c

Figure 3.36

£D *

"S’

S’
c

<D O

ss
&
O

(5

o.
og
</}

<D

I

o

o
CXI

0

.25

Fraction of the data

.75

104

Statistics with Stata

Quantile-normal plots, also called normal probability plots, compare quantiles of a
variable’s distribution with quantilesof a theoretical normal distribution having the same mean
and standard deviation. They allow visual inspection for departures from normality in every
part of a distribution, which can help guide decisions regarding normality assumptions and
efforts to find a normalizing transformation. Figure 3.37, a quantile-normal plot of energy,
confirms the severe positive skew that we had already observed. The grid option calls for
a set of lines marking the .05, .10, .25 (first quartile), .50 (median). .75 (third quartile), .90, and
.95 quantiles of both distributions. The .05. .50. and .95 quantile values are printed along the
top and right-hand axes.
. qnorm energy, grid
I

I

111.872

o
o

354.5

Figure 3.37

597.129

So
CO

■u’03

co

0)

E

0(0

I

O)

o o

ro
ro
o- o

8
co

CD CM
CL

CM

u_ o

o

0

200

400
Inverse Normal

600

800

Grid lines are 5,10.25. 50. 75. 90. and 95 percentiles

I

I!

•K''

i

Quantile-quantile plots resemble quantile-normal plots, but they compare quantiles
(ordered data points) of two empirical distributions instead of comparing one empirical
distribution with a theoretical normal distribution. On the following page. Figure 3.38 shows
a quantile-quantile plot of the mean math SAT score versus the mean verbal SAT score in 50
states and the District of Columbia. If the two distributions were identical, we would see points
along the diagonal line. Instead, data points form a straight line roughly parallel to the
diagonal, indicating that the two variables have different means but similar shapes and standard
deviations.

Graphs

105

• qqplot msat vsat

Figure 3.38

Quantile-Quantile Plot

o
co

S

0) io

o
w

£
co
E 1



^8
o

400

450
500
Mean verbal SAT score

550

Regression with Graphics (Hamilton 1992a) includes an introduction to reading quantilebased plots. Chambers et al. (1983) provide more details. Related Stata commands include
pnorm (standard normal probability plot), pchi (chi-squared probability plot), and qchi
(quantile-chi-squared plot).

I

I

Quality Control Graphs
Quality control charts help to monitor output from a repetitive process such as industrial
production. Stata offers four basic types: c chart, p chart, R chart, and x chart. A fifth type
called Shewhart after the inventor of these methods, consists of vertically-aligned x and R
charts. Iman (1994) provides a brief introduction to Rand x charts, including the tables used
in calculating their control limits. The Base Reference Manual gives the command details and
formulas used by Stata. Basic outlines of these commands are as follows:
.

cchart defects unit

Constructs a c chart with the number of nonconformities or defects (defects) graphed
against the unit number (unit). Upper and lower control limits, based on the assumption
that number ofnonconformities per unit follows a Poisson distribution, appear as horizontal
lines in the chart. Observations with values outside these limits are said to be “out of
control.”
. pchart rejects unit ssize

Constructs a p chart with the proportion of items rejected (rejects / ssize) graphed against
the unit number (unit). Upper and lower control limit lines derive from a normal
approximation, taking sample size (ssize) into account. If ssize varies across units the
control limits will vary too, unless we add the option stabilize

106

.

Statistics with Stata

rchart xl x2 x3 x4 x5, connect(1)

2)nSt^ttSian Rlrange) Chr “Smg the replicated measurements in variables xl through
’ ‘n
eXample’ fiVC reP'icat>ons per sample. Graph the range with^eaS
taga'nSt.the Sa^ple number’ a^ (optionally) connect successive ranges with line
egmonts. Horizontal lines indicate the mean range and control limits. Control limits are
estimated from the sample size if the process standard deviation is unknown When o is
known we can include tins information in the command. For example, assuming o = 10,

.

rchart xl x2 x3 x4 x5, connect(1)

std(10)

xchart xl x2 x3 x4 x5, connect(1)

S’crXih/ (mean)hChart asingthe rePl*cated measurements in variables xl through
• 5. Graphs the mean with in each sample against the sample number and connect successive
means with line segments. The mean range is estimated from the mean of sample means
and control limits from sample size, unless we override these defaults. For example if we
know that the process actually has |i = 50 and o =10,
P


xchart xl X2 x3 x4 x5, connect(l) mean(50)

std(10)

Alternatively, wetcould specify particular upper and lower control limits:
. xchart xl x2 x3 x4 x5, connect(l) mean(50)
upper (60)



shewhart xl x2 x3 x4 x5, mean(50)

lower(40)

std(10)

In one figure, vertically aligns an x chart with an Rchart.

To illustrate a p chart, we turn to the quality inspection data in quality1.dta.
Contains data from C:\data\qualityl.dta
obs:
16
vars:
3

112 (99.9* of memory free)
•■= - - able

= ~e

storage
type

display
format

byte
byte
byte

* ?. Og
* 3. Og
*5. Og

value
label

Quality control example 1
4 Jul 2005 12:07

variable label

Day sampled
Number of units sampled
Number of units rejected

Sorted by:

.

list in 1/5
I

day

ssize

rejects I

------ !
1. I
2. I
3. I
4. I
5. I

58

26
21
6

53
53
52

52
51

10
12
12
10
10

|
|
|
|
|

^hNOt^ ?atIlainPuIe SiZe VarieS from Unit t0 unit’ and that the units (days) are not in order

M£3a

Graphs

107

■ pchart rejects day sslze

Figure 3.39
•'T

■So
■o

s

<D
IO
O

ro
tr

CM

O

0

20
Day sampled

40

60

2 units are out of control

— serves to illustrate rchart and
, Dataset quality2.dta, borrowed from Iman C
(1994:662),
xchart . Variables xl through x4 represent repeated measurements from an industrial
production process; 25 units with four replications each form the dataset.
Contains data from C:\data\quality2.dta
obs:
25
vars:
4
size:
500 (99.9% of memory free)

variable name

Ii

storage
type

xl
x2
x3
x4

float
float
float
float

display
format

%9.0g
%9.0g
%9.0g
%9.0g

Sorted by:

. list in 1/5

1.
2.
3.

4.
5.

+
xl
I
I
I 4.6
I 6.7
I 4 .6
I 4.9
I 7.6

x2

x3

2
3.8
4.3
6
6.9

4
5.1
4.5
4.8
2.5

x4 |
I
3.6 I
4.7 I
3.9 I
5.7 I
4 .7 I

value
label

Quality control (Iman 1994:662)
4 Jul 2005 12:07

variable label

108

Statistics with Stata
i'r

Figure 3.40, an rR chart,

graphs variation in the process range over the 25 units, rchart
informs us that one unit’s range is “out of control.”
rchart xl x2 x3 x4, connect(1)

.

g

<o

Figure 3.40

co

a>
cn

S
02

CM
■MCM

CM

0

5

10
Sample

15

20

25

1 unit is out of control

Figure 3.41, an x chart, shows variation in the process mean. None of these 25 means falls
outside the control limits.
. xchart xl x2 x3 x4, connect(1)

CO

co

3

rco
co

o
cn
ro

<

CM

s
co

co

0

25^

5

15

10

Sample
0 units are out of control

20

Figure 3.41


Graphs

f-

109

Adding Text to Graphs

1

Titles, captions, and notes can be added to make graphs more self-explanatory. The default
versions of titles and subtitles appear above the plot space; notes (which might document the
data source, for instance) and captions appear below. These defaults can be overridden, of
course. Type help title_options for more information about placement of titles, or
help textbox_options for details concerning their content. Figure 3.42 demonstrates
the default versions of these four options in a scatterplot of the prevalence of smoking and
college graduates among U.S. states, using statehealth.dta. Figure 3.42 also includes titles for
both the left and right y axes, yaxis (1 2), and top and bottom .r axes, xaxis (1 2)
Subsequent ytitle and xtitle options refer to the second axes specifically, by
including the axis (2) suboption, y axis 2 is not necessarily on the right, and.v axis 2 is not
necessarily on the left, as we will see later; but these are their default positions.
. graph twoway scatter smokeT college, yaxis(1 2) xaxis (1 2)
title("This is the TITLE") isubtitle

("This is the SUBTITLE")
caption
("This is the CAPTION") note("This is the NOTE")
caption("This
ytitle("Percent adults smoking")
ytitle("This is Y AXIS 2", axis(2))
xtitle("Percent adults with Bachelor's degrees or higher")
xtitle ("This is X AXIS 2", axis(2))
Figure 3.42

This is the TITLE
10

15

This is the SUBTITLE
This is X AXIS 2
20
25

30

35
in

co

co

CT)

CO Csi

E
v)

to
X
m<

sro ™

% :



<0

o x:

o 04

cn h

Q

Q_

in

m
10

15
20
25
30
Percent adults with Bachelor's degrees or higher

35

This is the NOTE

This is the CAPTION

Titles add text boxes outside of the plot space. We can also add text boxes at specified
coordinates within the plot space. Several outliers stand out in this scatterplot. Upon
investigation, they turn out to be Washington DC (highest college value, at far right), Utah
(lowest smokeT value, at bottom center), and Nevada (highest smokeT value, at upper left).
Text boxes provide a way for us to identify these observations within our graph, as
demonstrated in Figure 3.43. The option text (15.5 22.5 "Utah”) places the word
“Utah” at position^ =15.5, x = 22.5 in the scatterplot, directly above Utah’s data point.

er 5‘ M3 (to H
3> *t)
er
§ rt
—’ 2
v 2
<P o
3 ft c r o
?
n*
g.
(D

3
3
CX
co

£> g

»<

'• hQ

C
3 31
Cu 3
M.

EE. 3
CD

8 o 3
3

<—►

0)

-i



s

3
co

a S <s3o'Cg£<

S’
3-

3‘

r-t X
CO
o’
3
3* *O

QQ

f-

CD

< CD
P
__CD CX co
cd < BL S' 3
3 w s2z a 3
CD
3 sC

a

3 UQ



5 £;
3
-U
iQ
o* £
£ CO £ 21 N

CD*

S ’2. O cr CfQ

g c *8

2

3

S
<? =*
to
co EF
>-—« CD
~

»■

ft

3 ~ S
3 = w
S S’ §
O S
co

. c 8? ? C CD
ST
i 2 2. »
; g 8> 2 =r’6

■ g-'H

C/1

CO

8 a co



s-co« Mj
p. 3 CD
3
rt
< 3
I o “ s
□ Er
3 S'
*u 3 £2. o
* • §-..O 5 S
- to E XJ <-t
&

oq

3

3‘

cT co “S 3 O 3
8 to Xo
R8 . o
CD
3
O 8- K 5 CD
3
o”2. „

Is I



23: •" *3o
S
3 S' 3^

Cu
CD
3
CD 3
CD ►—

G>


r

15

Q

Percent adults smokinq
20
25
30
35

O ■

£££'x*‘<‘<c.
O

o Q CD

Z

0)

CD

gen’
r-^

3
Q.
C_

•«


«



£0
Q.
0)

CH

w
^ND

-N5!^

CD

w'
W‘ ? w0
X
zr

3
g.

CD
O

3.

£ CDCco_ Hco

o.to_
CD
§
3
3
W
O

m m

ETG0
<qO
ET
3

. GO
O

£=53
Q) O (/>
*3^

*2.
4

o
Q

GO
CH 1

15

t£»y2W

ft (t (U
p- p- to

GO
CD

35

2!

c5‘
c
Q
W

4

GO

..
ft ,,
ft

H H R. o
~

Q
3 3
E3

=s

H §
~ H *

5~2.

to

cd

o

co
ET ID H P- W
ft U) p* ND p. bf
P- H EC
Ef co
co a
H- **
ND 'cn ND
**
:= COu q w an p3
(D
n> co h- rt
= to p- 5 H- 3
CO ft
M ft co Ct H0 53
(D
3 3 3
CO ft p
rt co <a x 3
K
3
EC
p.
1‘■ tr
“ P> ft
-*
CL
ID CO
o H- CL
~ 3 > C > CQ. ft
Ef
ej :5 3 ET X H X H (D
— iq
="
= h ft h rt
H ft > *■* to co to
- -w Q H 3
3 0
> to
th ej er
M « to to
rt
o
H
s 3 H = fj
— ox
O »•
a
X* O

= 3
B
p
H- Z 3 p.
CM
3
X to X EJ
2
c 3
h
p- ju k iq ~= er
= h
cq
iQ P- tQ
CO O co
ft
3
ET -2
P- CO p.
EJ
EJ
3
ND ID K>
O ft
—* EJ
ft H PJ
co 0
_ (0
..
— o —
3 3 3
3 ft 3
h
— <-x x
3
PJ
H 3
w
H H to
H Q
EC EC
— rt
CL
C
(D
co co
tr 3
cQ
ND
M» P*
p. p.
H
o P*
(D
co co
o »<
(D
X
CO
ft ft 3
O
»* £t X
M
=
o
(D (D P3
h
to
Z
to
ED* co
ET
o
a
p*
H- Ct
h w
ft 3
cQ
M H ND
CD Ct
EC
= H
(D
(D
H

5

h

(t EC

ft ft ft H $

c tn in cn H to H h3

WND

o
r*
(/)

CD (D 3 ft ft
XXX p- pD* rt rt rt
ef J2
o
H
ND U) p* (D (D
CO UD to —>

Z

3-0

o
S

N
3

«•
ro”
5
O z T)

>
TJ
H
O

co
S' SCD s g-’a. CD cz>
CD 5*
r.
r
2
CD
H. o QJA ex
?
8c g. g3Q H(D 2.
g
..
H
3
O
5
£
o
K.
CL 3
e. '£
a
■■*■ o
P o 3 CD
a a o 3 3 p‘ X5

IQ

o

v>
w

s>
5
v<‘

1 3

<:
c

O

£ 8 £
Q- « 5
CD
P J2. CO 2. S'
co
CD

-J

<
o

s

n

I—

O

Q er
co
CD
CD

rr

3-

H 3
in cd
p. CD

? s< h; f a

-3
CD
X- CD
‘ S rg
tt
3
OQ tt
ft
g co
co
£ # rt
3
3 TS ~
3 CD XJ
H

1

CD

3

£h $
CD- O 0 Q.


2. a
| 2.

co
ST
5?

??

to

Er
GO

s3

et£-

£

3
<__ 3
tt
3* ‘E'c
rt

S3
2
- CD
0 CD
55 CO co 75 O <

X

?

3
-■ S- §3c s.
92 a
& a
& £%
~d £ st
2c
S’ -S GO
CD
3

C. S’
EJ1 3

£ a o & 2. 3 U»II
ft


2 3

3
tt

co


Q-

0
O
-O 3
EF
3CD2 S

«
co g
3^ CO CO

3
er cd

5 2.

o V &


ET 3

(D

H U
Q



a er H
o o CD
X

ZTl
CD L»J
CD

Q- to-'p’

S* 3
3
3
to g
O p

3

“•
co
O CD CD
3 3
co CD o
3^

o

.. S' £

o
H £ CD
to o
- <
rt
3

II

H 3
3
3*

3

CD

QCu

3

3*
CD 3
co
CT
O O

x

?a
o- CD >.
< 3 —
’-L. E‘
=. 5
ET
CX a
C 3 c
co co

H
% s ?r - • 3
(D cz> CT
(T>

O
X

CD



5. 3> ~ co3
SBl
’ 5h: CD S*. I
CD 3
CD
S £ a
co

c
co

CO

3

t-

3
co

o

I

!

Graphs

111

. graph twoway Ifitci smokeT college
Figure 3.44

°
co J

</)
<D

zj m

_

TJ
<D

L.
O
v© o
o'' <\i
in N

cn

10

15

«
20
25
% adults college degree, 1990

95% Cl

----------

30

35

Fitted values

A more infonnative graph results when we overlay a scatterplot on top of the regression
line plot, as seen in Figure 3.45. To do this, we essentially give two distinct graphing
commands, separated by “ | | ”,
. graph twoway Ifitci smokeT college

I I

scatter smokeT college
Figure 3.45

m _l

D)C0

o

E
wo
5

■u
(Q

(D

J3
ro

H

ss wk

**

ft -

Is -

U-

o
m m zr

O)

15

10



20
25
% adults college degree, 1990

95% Cl
% adults smoking

----------

30

Fitted values

35

«!

112

Statistics with Stata

The second plot (scatterplot) overprints the first in Figure 3.45. This order has
consequences for the default line style (solid, dashed, etc.) and also for the marker symbols
(squares, circles, etc.) used by each sub-plot. More importantly, it superimposes the scatterplot
points on the confidence bands so the points remain visible. Try reversing the order of the two
plots in the command, to see how this works. •

Figure 3.46 takes this idea a step further, improving the image through axis labeling and
legend options. Because these options apply to the graph as a whole, not just to one of the
subplots, the options are placed after a second | | separator, followed by a comma. Most of
these options resemble those used in previous examples. The order (2 1) option here does
something new: it omits one of the three legend items, so that only two of them (2, the
regression line, followed by 1, the confidence interval) appear in the figure. Compare this
legend with Figure 3.45 to see the difference. Although we list only two legend items in Figure
3.46, it is still necessary to specify a rows (3) legend format as if all three were retained.
. graph twoway Ifitci smokeT college
I|
scatter smokeT college
I I
/ xlabel (12 (2)34) ylabel(14(2)32, angle (horizontal))
xtitie("Percent adults with Bachelor’s degrees or higher")
ytitle("Percent adults smoking")
note("Data from CDC and US Census")
legend(order (2 1) label (1 "95% c.i.") label (2 "regression line")
rows(3) position(l) ring(0))
Figure 3.46

32

regression line
95% c.i.

30
28
O)

IE 26
v>
JQ

24

-u

22

CD

<D

o 20
18
16
14

B

12

14

16
18
20
22
24
26
28
30
Percent adults with Bachelor's degrees or higher

32

34

Data from CDC and US Census

The two separate plots (If itci and scatter) overlaid in Figure 3.46 share the same
y and x scales, so a single set of axes applies to both. When the variables of interest have
different scales, we need independently scaled axes. Figure 3.47 illustrates this with an overlay
of two line plots based on the Gulf of St. Lawrence environmental data 'xngulfdtci. This figure
combines time series of the minimum mean temperature of the Gulfs cold intermediate layer
waters (czT), in degrees Celsius, and maximum winter ice cover (maxarea), in thousands of
square kilometers. The cil plot makes use of yaxis (1), which by default is on the left. The

Graphs

113

maxarea plot makes use of yaxis(2). which by default is on the right. The various
ylabel , ytitle , yline , and yscale options each include an axis(l) or
axis (2) suboption, declaring whichy axis they refer to. Extra spaces inside the quotation
marks for ytitle provided a quick way to place the words of these titles where we want
them, near the numerical labels. (For a different approach, see Figure 3.48.) The text box
containing “Northern Gulf fisheries decline and collapse” is drawn with medium-wide margins
around the text; see help marginstyle for other choices, yscale (range ( ))
options give both y axes a range wider than their data, with specific values chosen after
experimenting to find the best vertical separation between the two series.
. graph twoway line cil winter, ya-xis (1) yscale (range(-1,3) axis(l))
ytitle("Degrees C
", axis(1))
yline (0)
ylabel(-1(.5)1.5, axis(l) angle(horizontal) nogrid)
text(l 1992 "Northern Gulf" fisheries decline” "and collapse"
, box margin(medium) )
I|
line maxarea winter,
yaxis(2) ylabel(50(50)200, axis(2) angle(horizontal))
yscale(range(-100,221) axis(2))
ytitle ("
1000s of kmA2", axis (2))
yline(173.6, axis(2) Ipattern(dot))
I|
if winter > 1949,
xtitle("")
xlabel (1950(10)2000) xtick (1950 (2) 2002)
legend(position (11) ring(0) rows(2) order(2 1)
label (1 "Max ice area") label(2 "Min CIL temp"))
note("Source: Hamilton, Haedrich and Duncan (2003)- data from
DFO (2003)")
Figure 3.47

-- Min CIL tempi
— Max ice area

\
\

/

i
1 I 1 I
1/ ’ I
If I /
1/ ' /
I > I

1.5-J

/ /|,|/
/ /k

-200

Vvv\

I
/
l/\

\

04

-150E
cn

-100g

A
/

o

Northern Gulf

fisheries decline

U
w .5o

-50

<p

U)
<D

0

0

-.5-11950

1960

1970

1980

1990

2000

Source: Hamilton, Haedrich and Duncan (2003); data fromDFO (2003)

I
The text box on the right in Figure 3.47 marks the late- 1980s and early-1990s period when
key fisheries including the Northern Gulf cod declined or collapsed. As the graph shows, the
fisheries declines coincided with the most sustained cold and ice conditions on record.
X

114

Statistics with Stata

To place cod catches in the same graph with temperature and ice, we need three
independent vertical scales. Figure 3.48 involves three overlaid plots, with all y axes at left
(default). The basic forms of the three component plots are as follows:

connected maxarea winter
A connected-line plot of tnaxarea vs. winter, usingy axis 3 (which will be leftmost in our final
graph). The y axis scale ranges from -300 to +220, with no grid of horizontal lines. Its title is
‘Tee area. 1000 kmA2.” This title is placed in the “northwest" position, placement (nw).
line cil winter
A line plot of ciI vs. winter, using v axis 2. y scale ranges from -4 to +3, with default labels,

connected cod winter
A connected-line plot of cod vs. winter, usings axis I. The title placement is “southwest,”

placement(sw).

Bringing these three component plots together, the full command for Figure 3.48 appears
on the next page, v ranges for each of the overlaid plots were chosen by experimenting to find
the “right” amount of vertical separation among the three series. Options applied to the whole
graph restrict the analysis to years since 1959, specify legend and.v axis labeling, and request
vertical grid lines.

[

Graphs

115

graph twoway connected maxarea winter, yaxis(3)
yscale(range (-300,220) axis(3)) ylabel(50(50)200, nogrid axis (3))
ytitleflce area, 1000 km-2", axis(3) placement (nw) )
cipattern(dash)

II
line cil winter, yaxis(2) yscale(range(-4,3) axis(2))
ylabel(, nogrid axis(2))
ytitle("CIL temperature, degrees C", axis(2)) cipattern(solid)
I |
connected cod winter, yaxis(l) yscale(range(0,200) axis(l))
ylabel(, nogrid axis(l))
ytitle("Cod catch, 1000 tons", axis(l) placement(sw))

i

I I
if winter > 1959,
legend(ring (0) position(7)
label(2 "Min CIL temp")
i

xtitle("")

label(1 "Max ice area")
label(3 "Cod catch")) rows (3))
xlabel(1960(5)2000, grid)

Figure 3.48

CN

O
oO
ro
oo
o io

io

o T~
c/)
0)

O) •
<D
■o O

I(5 1?
oV
E
a>

—i

O

o

s

c
-2 o
o

A

\

ji

*

§o
X

_<----- Max ice area
-------- Min CIL temp
■g
OO

Cod catch

1960

1965

1970

1975

1980

1985

X

1990

1995

2000

Graphing with Do-Files
ompl icated graphics like Figure 3.48 require graph commands that are many physical lines
long (although Stata views the whole command as one logical line). Do-files, introduced in
Chapter 2, help in writing such multi-line commands. They also make it easy to save the
command for future re-use, in case we later want to modify the graph or draw it again.
The following commands, typed into Stata’s Do-file Editor and saved with the file name
fig03_48.do, become a new do-file for drawing Figure 3.48. Typing
.

do fig03_48

then causes the do-file to execute, redrawing the graph and saving it in two formats.

116

Statistics with Stata

^delimit

;

use c:\data•gulf.dta, c 1 e a r ;
graph twoway connected maxarea winter, yaxis(3)
•/scale (range (-300,220) < '
^^^50(50)200, nogrid axis(3))
ytitleCIce area, 1000 kmA2”,
, axis(3) placement (nw) )
s^pattern(dash)
II
line cil winter, yaxi‘s(2) yscale (range(-4, 3)
axis (2))
/label (, nogrid axis (2))
vtitle(”CIL temperature, degrees C", axiS(2)) cipattern(solid)
li
connected cod winter,
yaxis(l) ystale(ra.-.ge(0,200) axis(l))
ya.abel(, nogrid axis(l))
ytitle("Cod catch, 1000 tons", a x i s (11
p- acement(sw))
; I
if
> 2 95 9,
iegend(ring(0) position(7) labelfl "Max ---e area")
"Min CIL t-

.............

xlabel (1960 (5)2000, grid)
saving (c:\data\fig0 3_48.gph, replace) ;
graph export c:\data\fig03 48.eps replace ;
#delimit cr
~

treats this all as one logical line that ends with the semicolon after the saving () ontion
Tins option saves the graph in Stata’s .gph format.
9 '' P
'
Next, the graph export command creates a second version of the same eranh in
(Type^heS
h ‘ fOrmat’ 3S
by the epS Suff,x in the filename^ 48eps
Lf 1 r
P .graPh-exP°rt ‘o leant more about this command, which is partlcuZ
part,CularIy
useful for wntmg programs or do-files that will create graphs repeatedly.)

’ •fC d°
deinn,,go,ng back

cr command re-sets a carriage return as the end-of-Iine
Sla,a-s „s„al mode Alth()ugh , |s „„,;isib|e “ “

blanite.,.h7eZSX"“"

re“m<h‘'““

Retrieving and Combining Graphs
Any graph saved in Stata’s “live” .gph format
can subsequently be retrieved into memory by
the graph use command. For example, we
could retrieve Figure 3.48 by typing
.

graph use fi.g03__48

?d,‘iT™aPl, iS

“ is <lisp|wl»»»«" and c«» be printed or saved again with

saveeXZ
r
V'”1’ S"'d
EnhancedWindoXeXS” wT.riP‘

fo™>. “
^bfegnewly
’’'’"‘T
Ora|’l’iCS ( p”8’’ “

meniK nr dirArti • d
We a S0 COU d chan£e the coIor scheme, either through
mono I
d
i V
uhe graph use c°mmand. fig03 48.gph was saved in the s2
monochrome scheme, but we could see how it looks in the si colofCcheme by typing


1^

graph use fig03_47,

scheme(slcolor)

Graphs

117

Graphs saved on disk can also be combined by the graph combine command. This
provides a way to bring multiple plots into the same image. For illustration, we return to the
Gulf of St. Lawrence data shown earlier in Figure 3.48. The following commands draw three
simple time plots (not shown), saving them with the namesfig03_49a.gph,fig03_49b.gph, and
fig03_49c.gph. The margin (medium) suboptions specify the margin width for title boxes
within each plot.
. graph twoway line maxarea winter if winter > 1964, xtitlef ” )
xlabel (1965 (5)2000, grid) ylabel(50 (50) 200, nogrid)
title("Maximum winter ice area", position(4) ring(0) box
margin(medium))
ytitle("1000 kmA2") saving (fig03_49a)
• graph twoway line cil winter if winter > 1964, xtitle("")
xlabel (1965 (5) 2000, grid) ylabel(-1(.5)1.5, nogrid)
title("Minimum CIL temperature", position (1) ring(0) box
margin(medium))
ytitle("Degrees C") saving (fig03_49b)
. graph twoway line cod winter if winter > 1964 ,
xtitle (
)
xlabel (1965 (5) 2000, grid) ylabel(0 (20) 100, nogrid)
titie("Northern Gulf cod catch", position(l) ring(0) box
margin(medium))
ytitle("1000 tons") saving(fig03_49c)

To combine these plots, we type the following command. Because the three plots have
identical x scales, it makes sense to align the graphs vertically, in three rows. The imargin
option specifies “very small” margins around the individual plots of Figure 3.49.
. graph combine fig03__49a . gph fig03_49b . gph fig03_49c. gph,
imargin(vsmall) rows(3)

Figure 3.49

o

§S- \
s ■ _i_
1965

Maximum winter ice area
1970

1975

1980

1985

1990

1995

2000

Minimum CIL temperature

o'- ‘

SJ’Q 2ojo -

0)

Om

o

1965

1970

1975

1980

1985

1990

1995

2000

Northern Gulf cod catch

-

Is- __
§?o -

1965

1970

1975

1980

1985

1990

1995

2000

118

Statistics with Stata

details including the
h-C°"lbine for more information on this command. Options control
b on e sXr as thZ ’h 7, C°IUmnS’
°fa"d markers <which othe™se

Thev can Iso soec
t
P
the marginS betWeen individual P’°ts.
ex can also specify whether x ory axes of twoway plots have common scales or assign all
components a common color scheme. Titles can be added to the combined graph which can
be printed, saved, retrieved, or for that matter combined again in the usual ways

Our fina! example in^ratessevera! of'h^ graph combine optionS; and a
0
sn ot graP " n Uneqi'a'-S,zed comPonents. Suppose we want a scatteiplot similar to Ae
variable8
P Ot
earl'er 'n F'gUre 342’
with box PIots of thcT and x
x anables drawn beside their respective axes. Using statehealth.dta, we might first try to do this
box olot'of 3 '/Z
3 7iP 0fs,"okeT- a scatterplot of™IOA-eTvs. college, and a horizontal

50Sg



",g ‘l,e

Pk"S in,° “e im*sh«™>

• graph box smokeT, saving(wrongl)
. graph twoway scatter smokeT college, saving(wrong2)
graph hbox college, saving(wrong2)
. graph combine wrongl.gph wrong2. gph wrongj.gph

The combmed graph produced by the commands above would look wrong, however We
would end up with two fat box plots, each the size of the whole scatterplot, and none ofthe axes
ahgned. For a more sat.sfactory version, we need to start by creating a thin vertical box plot
7 onoz r
(20) option in the following command fixes the plot’s x (horizontal)
size at :0/o of normal, resulting in a normal height but only 20% width plot. Two empty
caption lines are included for spacing reasons that will be apparent in the final graph.
. graph box smokeT, fxsize(20) caption ("”
•’”)
ytitlef") ylabel(none) ytick (15 (5) 35 , grid) saving (fig03_50a)

For the second component, we create a straightforward scatterplot of smokeT vs. college.
graph twoway scatter smokeT college,
ytitle("Percent adults smoking")

xtitle("Percent adults with Bachelor’s degrees or higher")

ylabel (, grxd)

xlabel (, grid)

saving(fig03_5Ob)

Z .compon1e"t is a thin horizontal box plot ofcollege. This plot should have normal

X deluded’5,26

2°%°fnOrma1' F°rSp3CingreaS°nS’ tW°empty lefttitles

graph hbox college, fysize(20) lltitle("" ) 12title( H H J
ylabel(none) ytick(10(5)35, grid) ytitle("")
saving(fig03_50c)

Graphs

119

These three components come together in Figure 3.50. The graph combine
command’s cols (2) option arranges the plots in two columns, like a 2-by-2 table with one
empty cell. The holes (3) option specifies that the empty cell should be the third one, so
our three component graphs fill positions 1, 2, and 4. xscale(1.05) enlarges marker
symbols and text by about 5%, for readability. The empty captions or titles we built into the
original box plots compensate for the two lines of text (title and label) on each axis of the
scatterplot, so the box plots align (although not quite perfectly) with the scatterplot axes.
graph combine fig03_50a.gph fig03_50b.gph figO3_5Oc.gph,

cols(2) holes(3) iscale(1.05)
Figure 3.50

io

CD

co



1
1


</)
3 CN

•u
<TJ

O

<5

<

•••

*

•«

Ql lo

12
.
20
25
30
35
Percent adults with Bachelor’s degrees or higher

X__________ _____

-

*

: ;

Summary Statistics and Tables

statistics are available through the command tahc^t ^exlble arrangements of summary

Ce S C°ntain statistics such as
one’variabIe procedures

frequencies, sums, means or median, F
including normality tests transfoimation, ’^dd

are not described in this chapter, but

Example Commands
. summarize yl y2 y3

• summarize yl y2 y3, detail

Obtains detailed summary statistics mvh
including percentiles, median, mean, standard
deviation, variance, skewness, and kurtosis.


summarize yl if xl > 3 & x2 <

°"'V

ObSe™,iOnS

V™b,e

■ summarize yl [fweight = w], detail

Cafato detailed s„,™ary s.a.lsfa forj7 „sing ,he frequei,cy wc|gh<s jn


* *

y1'

Stats<mean =d skewness kurtosis n)

Calculates only the specified summary statistics for variableyl.
'

!0

Stat?(min P5 P25 p50 P75 P95 “ax) by(xl)

is

Summary Statistics and Tables

121

tabulate xl

Displays a frequency distribution table for all nonmissing values of variable xl.
.

tabulate xl,

sort miss

Displays a frequency distribution of.v/. including the missing values. Rows (values) are
sorted from most to least frequent.
.

tabl xl x2 x3 x4

Displays a series of frequency distribution tables, one for each of the variables listed.
.

tabulate xl x2

Displays a two-variable cross-tabulation with.v/ as the row variable, and.v? as the columns.
. tabulate xl x2, chi2 nof column

Produces a cross-tabulation and Pearson
test of independence. Does not show cell
frequencies, but instead gives the column percentages in each cell.
tabulate xl x2, missing row all

Produces a cross-tabulation that includes missing values in the table and in the calculation
of percentages. Calculates “all” available statistics (Pearso:
(Pearson and likelihood x:, Cramer’s
of
K Goodman and Kruskal’s gamma, and Kendall’s rb).
.

tab2 xl x2 x3 x4

Performs all possible two-way cross-tabulations of the listed variables.
tabulate xl,

summ(y)

Produces a one-way table showing the mean, standard deviation, and frequency of r values
within each category of xl.
tabulate xl x2, summ(y) means

Produces a two-way table showing the mean ofjy at each combination ofxl and x2 values.
. by x3, sort:

tabulate xl x2, exact

Creates a three-way cross-tabulation, with subtables for.tf (row) by.v2 (column) at each
value of.rd. Calculates Fisher’s exact test for each subtable, by varname, sort:
works as a prefix for almost any Stata command where it makes sense. The sort option
is unnecessary if the data already are sorted on vamame.
table y x2 x3, by(x4 x5) contents(freq)

Creates a five-way cross-tabulation, of r (row) by r’ (column) by x3 (supercolumn), by.v4
(superrow I) by x5 (superrow 2). Cells contain frequencies.
. table xl x2, contents(mean yl median y2)

Creates a two-way table of.v/ (row) by x2 (column). Cells contain the mean of vl and the
median ofy2.

Dataset VTtown.dta contains information from residents of a town in Vermont. A survey was
conducted soon after routine state testing had detected trace amounts of toxic chemicals in the
town s water supply. Higher concentrations were found in several private wells and near the
public schools. Worried citizens held meetings to discuss possible solutions to this problem.

H

122

Statistics with Stata

Contains data from C:\data\VTtown.dta
obs:
153
vars:
7
size:
1,683 (99.9% of
memory free)
variable name

storage
type

display
forma t

value
label

sexlbl

VT town survey (Hamilton 1985)
11 Jul 2005 18:05

variable label

gender
lived
kids
educ
meetings
contarn

byte
byte
byte
byte
byte
byte

%8.0g
* 8.0g
% 8.0g
8.0g
%8.0g
% 8.0g

kidlbl
con tamlb

school

byte

%8.0g

close

kidlbl

Respondent's gender
Years lived in town
Have children <19 in town?
Highest year school completed
Attended meetings on pollution
Believe own propertv'water
contaminated
School closing opinion

Sorted by:

To find the mean and standard deviation of the variable lived (years the respondent had
lived in town), type
summarize lived

.

|

Obs

Mean

lived |

153

19.26797

Variable

Std.

Dev.

Min

Max

16.95466

1

81

This table also gives the number of nonmissing observations and the variable's minimum and
maximum values. Ifwe had simply typed sununarize with no variable list, we would obtain
means and standard deviations for every numerical variable in the dataset.
To see more detailed summary statistics, type
.

summarize lived,

detail
Years lived in town

1%
5%
10%
25%

50%

75%
90%
95%
99%

Percentiles
1
2
3
5

Smallest
1
1
1
1

15

29
42
55
68

Largest
65
65
68
81

Obs
Sum of Wg£.

153
i5?

Mean
Std. De v.

19.26797
16.95466

Variance
Skewness
Kurtosis

287.4606
1.208804
4.025642

This summarize, detail output includes basic statistics plus the following:
Percentiles:
Notably the first quartile (25th percentile), median (50th percentile), and third
quartile (75th percentile). Because many samples do not divide evenly into
quarters or other standard fractions, these percentiles are approximations.
Four smallest andfour largest values, where outliers might show up.

»

Summary Statistics and Tables

123

Sum ofweights: Stata understands tour types of weights: analytical weights ( aweight).
frequency weights ( fweight ), importance weights ( iweight ), and
sampling weights (pweight). Different procedures allow, and make sense
with, different kinds of weights, summarize, detail , for example,
permits aweight or fweight. Forexplanations see help weights.
Variance’.
Standard deviation squared (more properly, standard deviation equals the
square root of variance).
Skewness:
The direction and degree of asymmetry. A perfectly symmetrical distribution
has skewness = 0. Positive skew (heavier right tail) results in skewness > 0;
negative skew (heavier left tail) results in skewness < 0.
Kurtosis:
Tail weight. A normal (Gaussian) distribution is symmetrical and has kurtosis
- 3. If a symmetrical distribution has heavier-than-normal tails (that is, is
sharply peaked), it will have kurtosis > 3. Kurtosis < 3 indicates lighter-thannormal tails.

The tabstat command provides a more flexible alternative to summarize. We can
specify just which summary statistics we want to see. For example,
.

tabstat lived,

stats(mean range skewness)

variable I

mean

range

s kewness

lived I

19.26'797

OA

1.208804

With a by(varname) option, tabstat constructs a table containing summary
statistics for each value of varname. The following example contains means, standard
deviations, medians, interquartile ranges, and number of nonmissing observations of lived, for
each category ofgender. The means and medians both indicate that, on average, the women
in this sample had lived in town for fewer years than the men. Note that the median column is
labeled “p50”, meaning 50th percentile.
tabstat lived,

stats(mean sd median iqr n)

Summary for variables: lived
by categories of: gender

by(gender)

f=e«r-rdert*s sender)

gender |

mean

sd

p50

iqr

N

male |
female |

23.48333
16.54839

19.69125
14.39468

19.5
13

28
19

60
93

|

19.26757

16.95466

15

24

153

Total

Statistics axailable for the stats () option of tabstat include:
mean
Mean
count

n
sum
max

Count of nonmissing observations
Same as count
Sum
Maximum

124

Statistics with Stata

min

Minimum

range

Range = max - min

sd

Standard deviation
Variance

var

cv

Coefficient of variation = sd / mean
semean
Standard error of mean = sd / sqrt(n)
skewness Skewness
kurtosis Kurtosis
median
Median (same as p50 )
Pl
iqr

1st percentile (similarly, P5 , pio , p25 , p50 , p75 . p95 , OF p99 )
Interquartile range = p75 - p25

q
Quartiles; equivalent to specifying p25 pso p75
Further tabstat options give control over the table layout and labeling. Type help
tabstat to see a complete list.

The statistics produced by summarize or f
tabstat describe the sample at hand. We
might also want to draw inferences about the population, for example, by constructing a 99%
confidence interval for the mean of lived'.
. ci lived, level (99)
Variable I

Obs

Mean

Std. Err.

[99% Conf. Interval!

lived |

153

19.26797

1.370703

15.69241

22.84354

Based on this sample, we could be 99% confident that the population mean lies somewhere
in the interval from 15.69 to 22.84 years. Here we used a level ( ) option to specify a 99%
confidence interval. If we omit this option, ci defaults to a 95% confidence interval.
Other options allow ci to calculate exact confidence intervals for variables that follow
binomial or Poisson distributions. A related command, cii , calculates normal, binomial, or
Poisson confidence intervals directly from summary statistics, such as we might encounter in
a published article. It does not require the raw data. Type help ci for details about both
commands.

Exploratory Data Analysis
Statistician John Tukey invented a toolkit of methods for exploratory data analysis (EDA),
which involves analyzing data in an exploratory and skeptical way without making unneeded
assumptions (see Tukey 1977; also Hoaglin, Mosteller, and Tukey 1983, 1985). Box plots,
introduced in Chapter 3, are one of Tukey’s best-known innovations. Another is the stem-andleaf display, a graphical arrangement of ordered data values in which initial digits form the
"stems” and following digits for each observation make up the "leaves.”

Summary Statistics and Tables

125

stem lived

Stem-and-leaf plot for lived (Years lived in town)
0*
0.
11.
2*
2.

I 1111111222223333333344444444
I 55555555555566666666777889999
I 0000001122223333334
I 55555567788899
I 000000111112224444
I 56778899
3’ i 00000124
3 . I 5555666789
4 * I 0012
4 . I 59
5* I 00134
5 . I 556
6*
6.
7*
7. I
8* I

5558

1

stem automatically chose a double-stem version here, in which 1* denotes first dibits
of 1 and second digits of 0-4 (that is, respondents who had lived in town 10-14 years), .
denotes first digits of 1 and second digits of 5 to 9 (15-19 years). We can control the number
of lines per initial digit with the lines ( ) option. For example, a five-stem version in which
the 1* stem hold leaves of 0-1, It leaves of2-3, If leaves of4-5, Is leaves of 6-7
QPld
1
lc*nirz*r> rip Q O
J L _ _ • . *
< •


leaves of 8-9 could be obtained by typing
.

stem lived, lines(5)

Type help stem for information about other options.
Letter-value displays (Iv) use order statistics to dissect a distribution.
lv lived
153

M
F
E
n
C
B
A
Z

77
39
20

I
I
I
10.5 |
5.5 |
3
I
2
I
1.5 |
1
I
I
I
inner fence I
outer fence |

i

Years lived in town

5
3
2
1
1
1
1
1

-31
-67

15
17
21
27
30.75
33
34.5
37.75
41

29
39
52
60.5
65
68
74.5
81

65
101

I
|
|
|
|
|
|
|
I
I
I
|
|

spread
24
36
50
59.5
64
67
73.5
80

pseudosigma
17.9731
15.86391
16.62351
16.26523
15.15955
14.59762
15.14113
15.32737

# below
0
0

# above
5
0

M denotes the median, and F the fourths (quartiles, using a different approximation than
the quartile approximation used by summarize, detail and tabsum). E D C
. . denote cutoff points such that roughly 1/8, 1/16, 1/32, . . . of the distribution remains
outside in the tails. The second column of numbers gives the “depth,” or distance from nearest
extreme, for each letter value. Within the center box, the middle column gives
midsummaries, which are averages of the two letter values. If midsummaries drift away from
the median, as they do for lived, this tefcus that.the.distribution becomes progressively more

126

Statistics with Stata

Finally, “pseudosigmas” in the Xht-hand mi F

S he aPProxlmate interquartile range,

be if these letter values described a Gaussian popuh?™ Th^F^ 'he.Standard deviation should
a "pseudo standard deviation" (PSD) oroGdZ
>eFpSeudoslgma> sometimes called
■PPr»™.Bnor„,.li,yi„s^^^
-"I »U.lie™istaM check for

1 • Comparing mean with median diagnoses overall skew­

mean > medtan
mean = median
mean < median

positive skew
symmetry
negative skew

2.
.......


standard deviation = PSD
standard deviation < PSD

normal tails
lighter-than-normal tails

r-ss™—*

..

lies outside the inner fenc^bm not omsid^'the'^ouZfen" :"" Va'Ue"

0UffiCr”’t

Th,
~ 3,QRss<F' - '■«&< or E, + l.5IQR<,iF +yiQR
The value of., ,s a -severe oudier if 1, „K ou,sMe lhe

x<Fl-3IQR or x> F . + 3IQR
1V gives these cutoffs and the number of outliers of each tvm e
the outer fences, occur sparsely (about two per million^ i^P '

°“tllers’values bey°nd

■^h). Sev.eou.lS"^



dis.ribXuo^.^Xbta.X,'0;'1™ r "W tos ” P’^

-wle

formal normal,ry rests, and

Normalit

Tests and Transformations

X'XeZSsXXX X “ vrbte tto “»»

normality, extending the graphical tools this^ °rat0Iy methods t0 check for approximate
quantile-normal plot!) preseme^'n ^p ^'1?^’
plOtS’
pl<^ a"d
skewness and X Uries iC»f rhe
evaluate the null hypothesis that
population.

co

i
i_ , ' detai1 ’ can more formally
mp e at hand came fo°m a normally-distributed

Summary Statistics and Tables

127

sktest lived
Skewness/Kurtosis tests for Normality

Variable I

Pr(Skewness)

Pr(Kurtosis)

0.0*00

0.028

lived |

adj chi2(2)

joint ----Prob>chi2

24.79

0.0000

sktest here rejects normality: lived appears significantly nonnormal in skewness (P =
.000), kurtosis (P = .028), and in both statistics considered jointly (P = .0000). Stata rounds off
displayed probabilities to three or four decimals; “0.0000” really means P < .00005.

Other normality or log-normality tests include Shapiro-Wilk IV( swilk ) and ShapiroFrancia II ( sfrancia ) methods. Type help sktest to see the options.
Nonlinear transformations such as square roots and logarithms are often employed to
change distributions shapes, with the aim of making skewed distributions more symmetrical
and perhaps more nearly normal. Transformations might also help linearize relationships
between variables (Chapter 8). Table 4.1 shows a progression called the “ladder of powers”
(Tukey 1977) that provides guidance for choosing transformations to change distributional
shape. The variable lived exhibits mild positive skew, so its square root might be more
symmetrical. We could create a new variable equal to the square root of lived by typing
. generate srlived = lived A.5

Instead of lived A . 5, we could equally well have written sqrt (lived)
Logarithms are another transformation that can reduce positive skew. To generate a new
variable equal to the natural (base e) logarithm of lived, type
. generate loglived = In(lived)

In the ladder of powers and related transformation schemes such as Box-Cox, logarithms take
the place of a “0” power. Their effect on distribution shape is intermediate between .5 (square
root) and -.5 (reciprocal root) transformations.
Table 4.1: Ladder of Powers

Transformation

Formula

Effect

cube

new = old A3

reduce severe negative skew

square

new = old A2

reduce mild negative skew

raw

old

no change (raw data)

square root

new = old A . 5

reduce mild positive skew

*°e<(or log10)

new = In(old)
new = loglO(old)

reduce positive skew

negative reciprocal root

new = -(old A-.5)

reduce severe positive skew

negative reciprocal

new = -(old A-l)

reduce very severe positive skew

negative reciprocal square

new = -(old A-2)

negative reciprocal cube

new = -(old A-3) r

ff

When raising to a power less than zero, we take negatives of the result to preserve the
original order — the highest value of old becomes transformed into the highest value of/?eiv,

128

Statistics with Stata

and so forth. When old itself contains negative or zero values, it is necessary to add a constant
before transformation. For example, ifarrests measures the number of times a person has been
attested (0 for many people), then a suitable log transformation could be
generate larrests = In(arrests + 1)

The ladder command combines the ladder of powers with sktest tests for
normality. It tries each power on the ladder, and reports whether the result is significantly
nonnormal. This can be illustrated using the severely skewed variable energy, per capita energy
consumption, from states.dta.
ladder energy

Transformation

f orr.” la

chi2 (2)

P(chi2)

cube
square
raw
squa re-root
log

ene rayA 3
energy"2
energy
sqrc(energy)
log (energy)
1 /sort(energy)
1/energy
1/(energyA2)
1/ (energy"3)

53.74
45.53
33.25
25.03
15.88
7.36
1 . 32
4.13
11.56

0.000
0.000
0.000
0.000
0.000
0.025
0.517
0.127
0.003

reciprocal root
reciproca1

reciprocal square
reciprocal cube

It appears that the reciprocal transformation, Menergy (or energy'1), most closely resembles
a normal distribution. Most of the other transformations (including the raw data) are
significantly nonnormal. Figure 4.1 (produced by the gladder command) visually supports
this conclusion by comparing histograms of each transformation to normal1 curves.
.

gladder energy

»•

cubic

o

i

square

8

i

§

g

f

0

o

02 OOa4»-06«&06<fr06eH)6e+09

0 ZOOOOOOOCKDOOSDOOOOOOOOt

sqrt
'tn

<D
Q

o

1000

-.05 -.04

-.03

1/sqrt

s

g
o

10

ig

800

o

o

CD
O

600

400

g

g

o

200

log

2

!

1

8

o

R

T

£

Figure 4.1

identity

s

15

20

25

30

inverse

o

I

I
-lllii.

o

5

i

t)

o

5.5

6

6.5

7

1/square

wwShOs

8

-.005 -.004-.003 -.002-.001

-.07

-. 000023)002300 IS) OS 00e1C8Se-21

-.06

1/cubic

I

-1.50e-071.00e-075.00e-08

0

Per capita energy consumed, Btu
Histograms by transformation

Figure 4.2 shows a corresponding'set of quantile-normal plots for these ladder of powers
transformations, obtained by the “quantile ladder” command qladder . To make the tiny
?•*

!

Summary Statistics and Tables

129

plots more readable in this example we scale the labels and marker symbols up by 25% with
e scale (1 25) option. The axis labels (which would be unreadable andPcrowded) Je
suppressed by the options ylabel (none) xlabel (none).

■ gladder energy, scale(1.25) ylabel(none) xlabel(none)

cubic

square

identity

sqrt

tog

1/sqrt

inverse

1/square

1/cubic

Figure 4.2

Per capita energy consumed, Btu
Quantite-Normal plots by transformation

(^for .he BoS

““ bCSke”° finds
y^)

or

{yA- 1}/A

A > 0 or A

0

yW = lnO)

A=0
such that y (A) has approximately 0 skewness.
Applying this to energy, we obtain the
transformed variable benergy.
. bcskewO benergy = energy, level (95)

I

Transform |

L

(energyAL—1)/L |
-1.246052
(1 missing value generated)

[95% Conf. Interval1

Skewness

-2.052503

. 000281

-.6163383

syDm^(asMne^XSke»„;s's«s*))T^

I

from our ladde.-of-powem eho.ee, .he

pjr. The ShSeXZ™L’a

" “ n“,

—2.0525 < A < —.6163
reje? SOmn
P°Ssibilities incIudi"g logarithms (A = 0) or square roots (A = 5)
Chapter 8 descnbes a Box-Cox approach^to regression modeling.
re roots (A .5).

130

Statistics with Stata

Frequency Tables and Two-Wa

Cross-Tabu lation s

categorical variables requite

-iZ? p

P°"ution P™1*"' »>

labulaiing rhe caregorL v.ri.Me
tabulate meetings
Attended |
meetings on |
pollution |

Freq.

Percent

Cum.

no |
yes |

106
47

69.28
30.72

69.28
100.00

Total |

153

100.00

tabulate canmTnTCeJrerUenCy distributions for variables that have thousands of values,
To construct a
however, you might fSt wanX^^p^ose'values'b/wlv3™1316
ma"y Va'UeS’

«cod. or .otocodo options (see Chapter l^or^el^pW“1' "

TTr'" ■

19 living in town):

»™«bul..io„. F„r

meetings by kids (whether respondent has children under

tabulate meetings kids

Attended |
meetings | Have children <19 in
on |
town?
pollution |
no
yes |

Total

no |
yes |

52
11

54 |
36 |

106
47

Total |

63

90 |

153

The first-named variable forms the
only 11 of theses neT^le’
fOrmS C°Iumns in the resulting table.
We see that
1
/
non-parents who attended the meetings
tabulate
'"1'has
h~3 3 number of options that are useful with frequency tables:
all

.hat borh eanabies have ordered categories, whereas ehi2 . 11:chi2 , a„d™
cchi2

Ways rhe ooarriburion

cell

Shows total percentages for each cell.

chi2

Pearson

clrchi2

Displays the contribution to likelihood-ratio r in each cell of a two-way table.

Pearson x‘ (ehi.squ.ted) i„ each cell „f a

test of hypothesis that row and column variables are independent


Summary Statistics and Tables

column
exact

131

Shows column percentages for each cell.
Fisher’s exact test ofthe independence hypothesis. Superiorto chi2 ifthe table
contains thin cells with low expected frequencies. Often too slow to be practical
in large tables, however.

expected Displays the expected frequency under the assumption of independence in each cell
of a two-way table.

I

gamma

Goodman and Kruskal’s y (gamma), with its asymptotic standard error (ASE).
Measures association between ordinal variables, based on the number of
concordant and discordant pairs (ignoring ties). -I < y < 1.

generate (new) Creates a set of dummy variables named newl, new2, and so on to represent
the values of the tabulated variable.
lrchi2

Likelihood-ratio %2 test of independence hypothesis. Not obtainable ifthe table
contains any empty cells.

matcell (matname) Saves the reported frequencies in matname.

matcoi (matname) Saves the numeric values ofthe 1 x c column stub in matname.

matrow (matname) Saves the numeric values ofthe r x 1 row stub in matname.
missing

Includes missing” as one row and/or column of the table.

nofreq

Does not show cell frequencies.

nokey

Suppresses the display of a key above two-way tables. The default is to display the
key if more than one cell statistic is requested and otherwise to omit it. Specifying
key forces the display of the key.

nolabel

Shows numerical values rather than value labels of labeled numeric variables.

plot

Produces a simple bar chart of the relative frequencies in a one-way table.

replace

Indicates that the immediate data specified as arguments to the tabi command
are to be left as the current data in memory, replacing whatever data were there.

row

Shows row percentages for each cell.

sort

Displays the rows in descending order of frequency (and ascendine order ofthe
variable within equal values of frequency).

subpop (varname) Excludes observations for which varname = 0 in tabulating frequencies,
The identities of the rows and columns will be determined from all the data,
including the varname 0 group, and so there may be entries in the table with
frequency 0.

A

taub

Kendall’s r b ((tau-b), with its asymptotic standard error (ASE). Measures
association between ordinal variables. taub is similar to gamma , but uses a
correction for ties. - 1 < t b < 1.

V

Cramer’s V (note capitalization), a measure of association for nominal variables.
In 2 x 2 tables,' - IIn< larger
KI.tables, 0 < V < 1.

.Ur

132

Statistics with Stata

wrap

Requests that Stata take no action on wide, two-way tables to make them readable.
Unless wrap is specified, wide tables are broken into pieces for readability.

To get the column percentages (because the column variable, kids, is the independent
vanable) and a X2 test for the cross-tabulation of meetings by kids, type
.

tabulate meetings kids,

column chi2

I Key
|
|------------------ ,
I
frequency
|
I column percentage |
Attended |

meetings | Have children <19 in
on I
town?
pollution |
no
yes |
----- +
no |
52
54 |
I
82.54
60.00 |
----- +
yes |
I

Total |

I

Total

106
69.28

11
17.46

36 |
40.00 |

47
30.72

63
100.00

90 |
100.00 |

153
100.00

=

Pearson chi2(l)

8.8464

Pr = 0.003

percent of tIie respondents with children attended meetings, compared with about
/ /o ofthe respondents without children. This association is statistically significant (P=.003).
Occasionally we might need to re-analyze a published table, without access to the original
raw data. A special command, tabi (“immediate” tabulation), accomplishes this. Type the
cell frequencies on the command line, with table rows separated by “ \ For illustration, here
is
how tabi
----— could reproduce the previous %2 analysis, given only the four cell frequencies:
.

tab! 52 54

\ 11 36,

column chi2

I Key
I
|--------------------------------------(

I
frequency
|
I column percentage |

I
row |
--- +

1

2 I

Total

i I
I

52
82.54

54 |
60.00 |

106
69.28

2 I
I

11
17.46

47
30.72

Total |
I

63
100.00

36 |
40.00-1
------ +
90 |
100.00 |

col

Pearson chi2(l)

=

8.8464

153
100.00
Pr = 0.003

'I

Summary Statistics and Tables

133

Unlike tabulate, tabi does not require or refer to any data in memory. By adding the
replace option, however, we can ask tabi to replace whatever data are in memory with
he new cross-tabulation. Statistical options ( chi2 , exact, nofreq, and so forth) work
the same for tabi as they do with tabulate .

Multiple Tables and Multi-Way Cross-Tabulations
With surveys and other large datasets, we.sometimes n&
^ed frequency distributions of many
different variables. Instead of asking for each table :
separately, for example by typing
tabulate meetings, then tabulate gender, and finally tabulate kids
, we
could simply use another specialized command, tabi :
.

tabi meetings gender kids

Or to produce one-way frequency tables for each variable from gender through school in this
dataset (the maximum is 30 variables at one time), type
-

tabi gender-school

Similarly, tab2 creates multiple two-way tables. For example, the following command
cross-tabulates every two-way combination of the listed variables:
.

tab2 meetings gender kids

tabi and tab2 offer the same options as tabulate .

To form multi-way contingency tables, one approach uses the ordinary tabulate
command with a by prefix. Here is a three-way cross-tabulation of meetings by kids by
c°nta'>‘ (respondent believes his or her own property or water contaminated), with y2 tests for
the independence of meetings and kids within each level of contam'.
■ by contam,

sort:

tabulate meetings kids, nofreq col chi2

->
Attenaea |
reetmos : Have children <19 in
town?
poll t ion
no
yes I

Total

no |
yes i

91.30
8.70

68.75 |
31.25 |

78 . 18
21.82

Total |

100.00

100.00 |

100.00

Pearson chi2(l) =

7.9814

Pr = 0.005

134

Statistics with Stata

contam = yes

Attended
meetings
on
pollution

|

i
|

-------- -

no |
yes |
Total

Have children <19 in
town ?
no
yes |

58.82
4 1.18

38.46 |
61.54 |

46.51
53.49

100.00

100.00 |

100.00

-------- -

|

Total

Pearson chi2(l)

=

1.7131

Pr = 0.191

arents were more like y to attend meetings, among both the contaminated and uncontaminated
groups. On y among the larger uncontaminated group is this “parenthood effect" statistically
significant however As multi-way tables separate the data into smaller subsamples, the size
of these subsamples has noticeable effects on significance-test outcomes.
This approach can be extended to tabulations of greater complexity. For example to get
a our-way cross-tabulation of gender by contam by meetings by kids, with X2 tests for each
meetings by kids subtable (results not shown), type the command
.

by gender contam,

sort:

tabulate meetings kids,

column chi2

tableS’ifwed0 not need Percentages or statistical tests,
is through Stata’s general table-making command, table Tl.L
. This versatile command has many
options, only a few of which are illustrated here. To construct a simple frequency table of
meetings, type
table meetings,

contents(freq)

Attended
meetings
on
|
pollution |

Freq.

no |
yes I

106
47

For a two-way frequency table or cross-tabulation, type
.

table meetings kids,

Attended
meet:nq=
on
pollution
no |
yes I

Have
children
<19 in
town?
no
yes
52
11

54
36

contents(freq)

Summary Statistics and Tables

135

If we specifya third categorical variable, it forms the ■■supercolumns” of a three-way table:
table meetings kids content, contents (freq)

.

Attended
meet ings
on
pollution

I
I
I

no l

= e_ie-.-e own
property/water
contaminated and Have
children <19 in town?
-- nc
- yes -no
yes
no
yes
42

44

10
16

More complicated tables require the by ( ]
vari bles tabl tl
( } Opti°n’ Wh'Ch a,1°WS UPt0 four ‘WpetTOw”
variables,
one
y table thus can produce up to seven-way tables: c
---- row, one column one
supercolumn, and up to four superrows. ----Hereisisa afour-way
four-wavexample:
examole:
.

table meetings kids contain,

Responden |
I
gender
|I
and
Ii
Attended I
meetings
|I
on
|
pollution |

contents(freq) by(gender)

t 's

male

Believe own
property,/water
contaminated and Have
children <19 in town?
no-- -- yes __
yes
no
ves
no
yes

I
no |
ye s |

■18

2

1?
7

3
3

3
6

no
yes

24
2

26
13

7
4

7
10

female

The contents ( ) option of table
contents(freq)

specifies what statistics the table’s cells contain:
Frequency

contents(mean varname)

Mean of varname

contents(sd varname)

Standard deviation of varname
Sum of varname

contents(sum varname)

contents(rawsum varname)
contents(count varname)

I

contents(n varname)
contents(max varname)

contents(min varname)
contents(median varname)

I

contents(iqr varname)

Sums ignoring optionally specified weight
Count of nonmissing observations of varname
Same as count
Maximum of varname
Minimum of varname
Median of varname

Interquartile range (IQR) of varname

136

Statistics with Stata

contents (pl

varname)

contents (p2

varname)

1st percentile of varname

2nd percentile of varname (so forth to p99 )
The next section illustrates several more of these options.

Tables of Means, Medians, and Other Summary Statistics
ttb,U1^te 'Td'ly Pr°dUCeS tableS °f meanS and standard Aviations within categories of the

ssxx;,fon" * one-w‘y 'awe wi,h “"s °f wi,hi"
tabulate meetings,

Attended |
meetings on |
pollution |

summ(lived)

Summary of Years lived in town
Mean
Std. Dev.
Freq.

---------- -

no |
yes |

21.509434
14.212766

17.743833
13.911139

106
47

Total |

19.267974

16.954663

153

^th'^M vear^frS .T"3' h b" 7'a"Ve newcomers, averaging 14.2 years in town, compared
h ith 21.5 years for those who did not attend.
We can also use tabulate to form a two-way table of means by typing
tabulate meetings kids,

sum(lived)

means

Means of Years lived in town

Attended I
meetings I
on |
pollution |

Have children <19
in town?
no
yes |

no | 28.307692
yes I 23.363636
Iota1

!

27.444444

Total

14.962963 I 21.509434
11.416667 ; 14.212766
13.544444

(

19.267974

Both parents and nonparents among the meeting attenders tend to have lived fewer years in
refle^tiTn
SXlytoaJenT " SPUri°US
The means option used above called fora table containing only means. Otherwise we
get a bulkier table with means, standard deviations, and frequencies in each cell. Chapter 5
describes statistical tests for hypotheses about subgroup means.
Although it performs no tests, table nicely builds up to seven-way tables containing
means standard deviations, sums, medians, or other statistics (see the option list in previous
section). Here is a one-way table showing means of lived within categories of meetings:

F
Summary Statistics and Tables

137

table meetings, contents(mean lived)

Attended I
meetings
I
on
I
pollution I mean(lived)
no |
yes |

21.5094
14.2128

A two-way table of means is a straightforward extension:
.

table meetings kids,
contents(mean lived)

Attended
meetings
on
pollution
no
yes

I
(Have children <19
I
in town?
I
no
yes
I 28.3077
I 23.3636

14.963
11.4167

Table cells can contain more than one statistic,
Suppose we want a two-way table with both
means and medians of the variable lived:
.

table meetings kids, contents{mean lived
median lived)

Attended l
meetings
(Have children <19
on
I
in town?
pollution I
no
yes
no

I

yes

I 28.3077
I
27.5
I
I 23.3636
I
21

14.963
12.5
11.4167
6

I

I
I
I

The cell contents shown by table could be means, medians, sums
or other summary
statistics for two or more different variables.

138

Statistics with Stata

Using Frequency Weights
.S.“^rtZe ’ ,tabulate > table , and related commands can be used with frequency
weights that indicate the number of replicated observations
For example, file sextab2.dta

*Bri,ish !Urvey *•“*beh™ <«”•«

Ss

Contains data from C:\data\sextab2.dta
o b s:
48
vars:
4
size:
432 (99.9% of

memory free)

storage
type

variable name

age
gender
lifepart
count

byte
byte
byte
int

Sorted by:

age

display
format

value
label

%8.0g
%8.0g
%8.0g
%8.0g

age
gender
partners

1i fepart

British sex survey (Johnson 92)
11 Jul 2005 18:05

variable label
Age
Gender
# heterosex partners lifetime
Number of individuals

gender

16 “24’ ““

h«— XXplSrTS”vJ"B w“e
.

■»

list in 1/5

i.
2.
3.
4.
5.

+
I
age
I
I 16-24
I 16-24
I 16-24
I 16-24
I 16-24

gender

lifepart

count |

--------- ,
male
female
male
female
male

none
none
one
one
two

405
465
323
606
194

|
|
|
|
|

We use count as a frequency weight to create a cross-tabulation of lifepart by gender:
.

tabulate lifepart gender
--------- [fw = count]
# I

heterosex |
partners |
lifetime |

Gender
male
female |

Total

|
|
|
|
|
|

544
1734
887
1542
1630
2048

586
4146
1777
1908
1364
708

|
|
|
|
|
|

1130
5880
2664
3450
2994
2756

Total |

8385

10489 |

18874

none
one
two
3-4
5-9
10+

The usual tabulate options work as texpected' with
' * frequency
~
weights. Here is the same
table showing column percentages instead of frequencies:
A

--------- r-'

tv

AbJLA

JLJ.’

Summary Statistics and Tables

tabulate lifepart gender [fweight = count],

.

#
heterosex
partners
lifetime

I
|
|
|

none
one
two
3-4
5-9
10+

|
I
|
|
|
|

Total |

Gender
male
female

lotal

6.49
20.68
10.58
18.39
19.44
24.42

5.59
39.53
16.94
18.19
13.00
6.75

|
|
|
|

5.99
31. 15
14.11
18.28
15.86
14.60

100.00

100.00 |

100.00

139

column nof

Other types of weights such as probability or analytical weights do not work as well with
tabulate because their meanings are unclear regarding the command’s principal options.
A different application of frequency weights can be demonstrated with summarize. File
coliegel.dta contains information on a random sample consisting of 11 U.S. colleges, drawn
from Barron’s Compact Guide to Colleges (1992).
Contains data from C:\data\collegel.dta
obs:
11
vars:
5
size:
429 (99.9% of memory free)

variable name
school
enroll
pctmale
msat
vsat

I
I

storage
type

str28
int
byte
int
int

display
format

value
label

%28s
%8.0g
%8.0g
%8.0g
%8.0g

Colleges sample 1
11 Jul 2005 18:05

(Barren's

)

variable label
College or university
Full-time students 1991
Percent male 1991
Average math SAT
Average verbal SAT

Sorted by:

The variables include msat, the mean math Scholastic Aptitude Test score at each of the 11
schools.
.

list school enroll msat

1.
2.
3.
4.
5.

6.
7.
8.
9.
10 .
11 .

4L

tk.

I
school
I --I
3rown University
I
U . S c r a n t on
I U. North Carolina/Asheville
I
Claremont College
I
CePaul University
I—Thomas Aquinas College
I
Davidson College
I
I
U. Michigan/Dearborn
I
Mass. College of Art
Oberlin College
I
I---

enroll

American University

5228

I

5550
3821
2035
849
6197

201
1543
3541
961
2765

msat I
----- !
680
554
540 I
660 I
547 I
I
570 I
640 I
485 I
482 I
64 0 I
I
587 |

I

140

Statistics with Stata

We can easily find the mean msat value among these 11 schools by typing
.

I ,

summarize msat

Variable |

Obs

Mean

Std. Dev.

Min

Max

|

11

580.4545

67.63155

4 82

680

msat

This summary table gives each school’s mean math SAT score the same weight DePaul
University, however, has 30 times as many students as Thomas Aquinas College.^To take the
different enrollments into account we could weight by enroll.
.

summarize msat [fweight = enroll]
Variable |
----------------- + -_
msat |

!

Obs

Mean

Std. Dev.

Min

Max

32691

583.064

63.10665

4 82

680

Typing
.

summarize msat [freq = enroll]

would accomplish the same thing.

i
'i'i

I

I

f
I!

i

enroIIment’weighted mean, unlike the unweighted mean, is equivalent to the mean for
the 32,691 students at these colleges (assuming they all took the SAT). Note, however, that we
cou not say the same thing about the standard deviation, minimum, or maximum. Apart from
the mean, most individual-level statistics cannot be calculated simply by weighting data that
already are aggregated. Thus, we need to use weights with caution. They might make sense
in the context of one particular analysis, but seldom do for the dataset as a whole, when many
aitterent kinds of analyses are needed.

I___ II.I -JM1

ANOVA and Other
Comparison Methods

Analysis of variance (ANOVA) encompasses a set of methods for testing hypotheses about
differences between means. Its applications range from simple analyses where we compare the
means of v across categories of.v, to more complicated situations with multiple categorical and
measurement x variables, t tests for hypotheses regarding a single mean (one-sample) or a pair
of means (two-sample) correspond to elementary forms of ANOVA.
Rank-based “nonparametric” tests, including sign, Mann-Whitney, and Kruskal-Wallis,
take a different approach to comparing distributions. These tests make weaker assumptions
about measurement, distribution shape, and spread. Consequently, they remain valid under a
wider range of conditions than ANOVA and its “parametric” relatives. Careful analysts
sometimes use parametric and nonparametric tests together, checking to see whether both point
toward similar conclusions. Further troubleshooting is called for when parametric and
nonparametric results disagree.

I

anova is the first of Stata’s model-fitting commands to be introduced in this book. Like
the others, it has considerable flexibility encompassing a wide variety of models, anova can
fit one-way and AA-way ANOVA or analysis of covariance (ANCOVA) for balanced and
unbalanced designs, including designs with missing cells. It can also fit factorial, nested,
mixed, or repeated-measures designs. One follow-up command, predict , calculates
picdicted \alues. several types of residuals, and assorted standard errors and diagnostic
statistics after anova. Another followup command, test, obtains tests of user-specified
null hypotheses. Both predict and test work similarly with other Stata model-fitting
commands, such as regress (Chapter 6).

The following menu choices give access to most operations described in this chapter:
Statistics - Summaries, tables, & tests - Classical tests of hypotheses
Statistics - Summaries, tables, & tests - Nonparametric tests of hypotheses

Statistics - ANOVA/MANOVA
Statistics - General post-estimation - Obtain predictions, residuals, etc., after estimation

I

I
J.

(graphics - Overlaid twoway graphs

141

Ji I

142

Statistics with Stata

Example Commands
.

anova y xl x2

IfxJand ^O‘Way AN0VA’ tes,in2 for differences amongthe means ofy across categories
. anova y xl x2 xl*x2

Performs a two-way factorial ANOVA, including both the main and interaction (x/*x2)
effects of categorical variables x7 and x2.
.

anova y xl x2 x3 xl*x2 xl*x3 x2*x3 xl*x2*x3

Perf onns a three-way factorial ANOVA, including the three-way interaction x7 *x2*x3 as
well as all two-way interactions and main effects.
.

anova reading curriculum / teacher\curriculum

Fits 3 nested model to test the effects of three types of curriculum on students’ reading
ability (reading), teacher is nested within curriculum ( teacher] curriculum )
because several di fferent teachers were assigned to each curriculum. The Base Reference
Manual provides other nested ANOVA examples, including a split-plot design.
.

anova headache subject medication, repeated(medication)

Fits a repeated-measures ANOVA model to test the effects of three types of headache
medication (medication) on the severity of subjects’ headaches (headache). The sample
consists of 20 subjects who report suffering from frequent headaches. Each subject tried
each of the three medications at separate times during the study.
. anova y xl x2 x3 x4 x2*x3, continuous(x3 x4) regress

Perf onns analysis of covariance (ANCOVA) with four independent variables, two of them
(x7 and x2) categorical and two of them (x3 and x4) measurements. Includes the x2*x3
interaction, and shows results in the form of a regression table instead of the default
ANOVA table.
.

kwallis y, by(x)

Pert onns a Kruskal-Wallis test of the null hypothesis that y has identical rank distributions
across the k categories ofx (k > 2).
oneway y x

Performs a one-way analysis of variance (ANOVA). testing for differences among the
means ot y across categories of.v. The same analysis, with a different output table, is
produced by anova y x .
. oneway y x, tabulate scheffe

Perfonns one-way ANOVA. including a table of sample means and Scheffe multiplecomparison tests in the output.
.
I

ranksum y, by(x)

Performs a Wilcoxon rank-sum test (also known as a Mann-Whitney U test) of the null
ypothesis that y has identical rank distributions for both categories of dichotomous
variable x. If we assume that both rank distributions possess the same shape, this amounts
to a test for whether the two medians of are equal.

I

1

ANOVA and Other Comparison Methods

.

143

serrbar ymean se x, scale(2)

Constructs a standard-error-bar plot from a dataset of means. Variable ymean holds the
group means of r; se the standard errors; and x the values of categorical variable x.
scale (2) asks for bars extending to ±2 standard errors around each mean (default is ±1
standard error).
signrank yl = y2

Performs a Wilcoxon matched-pairs signed-rank test for the equality of the rank
distributions ofyl andy2. We could test whether the median ofyl differs from a constant
such as 23.4 by typing the command signrank yl = 23.4.
signtest yl = y2

Tests the equality of the medians of yl and y2 (assuming matched data; that is, both
variables measured on the same sample of observations). Typing sign test yl = 5
would perform a sign test of the null hypothesis that the median ofy/ equals 5.
.

ttest y = 5

Performs a one-sample t test of the null hypothesis that the population mean ofy equals 5.
ttest yl = y2

Performs a one-sample (paired difference) t test of the null hypothesis that the population
mean ofyl equals that ofy2. The default form of this command assumes that the data are
paired. With unpaired data (yl and y2 are measured from two independent samples), add
the option unpaired.
ttest y, by(x) unequal

Performs a two-sample t test of the null hypothesis that the population mean ofy is the
same for both categories of variable x. Does not assume that the populations have equal
variances. (Without the unequal option, ttest does assume equal variances.)

One-Sample Tests

Ii
I

I
I
I

One-sample t tests have two seemingly different applications:
1. Testing whether a sample mean y differs significantly from an hypothesized value p0.
2. Testing whether the means ofy and y2, two variables measured over the same set of
observations, differ significantly from each other. This is equivalent to testing whether the
mean of a “difference score” variable created by subtractingy1 fromy equals zero.
We use essentially the same formulas for either application, although the second starts with
information on two variables instead of one.
The data in writing.dta were collected to evaluate a college writing course based on word
processing (Nash and Schwartz 1987). Measures such as the number of sentences completed
in timed writing were collected both before and after students took the course. The researchers
wanted to know whether the post-course measures showed improvement.

TH

144

Statistics with Stata

describe
Contains data from C:\data\writing.dta
obs:
24
vars:
9
size:
312 (99.9% of memory free)

i

i

variable name
id
preS
prep
preC
preE
posts
post?
pos tC
pos tE

I

storage
type

byte
byte
byte
byte
byte
byte
byte
byte
byte

Nash and Schwartz (1987)
12 Jul 2005 10:16

display
format

value
label

variable label

%8.0g
%8.0y
%8.0g
%8.0g
%8.0g
%8.0g
% 8.0g
%8.0g
%8.0g

slbl

Student ID

# of sentences (pre-test)
. # of paragraphs (pre-test)
Coherence scale 0-2 (pre-test)
Evidence scale 0-6 (pre-test)
# of sentences (post-test)
# of paragraphs (post-test)
Coherence scale 0-2 (post-test)
Evidence scale 0-6 (post-test).

Sorted by:

Suppose that we knew that students in previous years were able to complete an average of
0 sentences. Before examining whether the students in writing.dta improved during the
course, we might want to learn whether at the start of the course they were essentially like
earlier students — in other words, whether their pre-test (preS) mean differs significantly from
the mean of previous students (10). To see a one-sample t test of
= 10, type
.

ttest preS = 10

One-sample t test
Variable

I

Obs

Mean

preS

24

10.79167

Decrees

freedom.:

Std. Err.
A O

O

«

Std. Dev.

[95% Conf. Interval]

4.606037

8.846708

12.73663

Ho: mean(preS) = 10

H= : rear.
t

<

il

t

10
u . ‘ ?oo

Ha: mean != 10
t =
0.8420
P > It | =
0.4084

Ha : mean > 10
t
0.8420
P > t =
0.2042

. Then°tftlon p >
means “the probability of a greater value oft ’’—that is, the one-tail
test probability. The two-tail probability of a greater absolute t appears as P > |t| =
.4084. Because this probability is high, we have no reason to reject H0:p = 10. Note that
ttesst automatically provides a 95% confidence interval for the mean. We could get a
different confidence interval, such as 90%, by adding a level (90) option to this command.

I

A nonparametnc counterpart, the sign test, employs the binomial distribution to test
hypotheses about single medians. For example, we could test whether the median of preS
equals 10. sign test gives us no reason to reject that null hypothesis either.

j

il
1;.

ANOVA and Other Comparison Methods

.

145

signtest preS = 10

Sign test

sign |

observed

expected

positive |
negative |
zero |

12
10
2

11
11
2

all |

24

24

One-sided tests:
Ho: median of preS - 10 = 0 vs.
Ha: median of preS - 10 > 0
Pr(#positive >= 12) =
Binomial(n = 22, x >= 12, p = 0.5) =

0.4159

Ho: median of preS - 10 = 0 vs.
Ha: median of preS - 10 < 0
Pr(#negative >= 10) =
Binomial(n = 22, x >= 10, p = 0.5) =

0.7383

Two-sided test:
Ho: median of preS - 10 = 0 vs.
Ha: median of preS - 10 != 0
Pr(#positive >= 12 or #negative >= 12) =
min(l, 2*Binomial(n = 22, x >= 12, p = 0.5)) =

0.8318

Like ttest, signtest includes right-tail, left-tail, andtwo-tail probabilities. Unlike
the symmetrical t distributions used by ttest, however, the binomial distributions used by
sign test have different left- and right-tail probabilities. In this example, only the two-tail
probability matters because we were testing whether the writing.dta students “differ” from their
predecessors.

I

I

Next, we can test for improvement during the course by testing the null hypothesis that the
mean number of sentences completed before and after the course (that is, the means ofpreS and
postS) are equal. The ttest command accomplishes this as well, finding a significant
improvement.
ttest posts = preS

Paired t test

I

Variable |

Obs

Mean

Std. Err .

Std. Dev.

[95% Conf. Interval]

posts |
preS |

24
24

26.375
10.79167

1 . 693779
.9402034

8.297787
4.606037

22.87115
8.846708

29.87885
12.73663

diff |

24

15.58333

1.383019

6.775382

12.72234

18.44433

Ho: mean(posts - preS) = mean(diff) = 0

Ha : mean (diff) < 0
t = 11.2676
P < t =
1.0000

Ha: mean (diff) != 0
t = 11.2676
P > It| =
0.0000 '

Ha : mean(diff) > 0
t = 11.2676
P > t =
0.0000

Because we expect “improvement,” not just “difference” between the preS and postS
means, a one-tail test is appropriate. The displayed one-tail probability rounds off four decimal

146

Statistics with Stata

places to zero (“0.0000” really means P< .00005). Students’ mean sentence completion does
l^^'ZXes^ °n thiS SamPle’ 3re 95% COnfldent that k imProves by between
t tests assume that variables follow a normal distribution. This assumption usually is not
critical because the tests are moderately robust. When nonnormality involves severe outliers
however, or occurs in small samples, we might be safer tumingto medians instead of means and
employing a nonparametnc test that does not assume normality. The Wilcoxon signed-rank
test, for example, assumes only that the distributions are symmetrical and continuous. Applying
a signed-rank test to these data yields essentially the same conclusion as ttest that
students’ sentence completion significantly improved. Because both tests agree on this
conclusion, we can assert it with more assurance.
.

signrank posts = preS

Wilcoxon signed-rank test
sign |

obs

sum ranks

expected

positive |
negative |
zero |

24
0
0

300
0
0

150
150
0

all |

24

300

300

unadjusted variance
adjustment for ties
adjustment for zeros

1225.00
-1.63
0.00

adjusted variance

1223.38

Ho: posts = preS
z =
Prob > Iz| =

4.289
0.0000

Two-Sample Tests

The remainder of this chapter draws examples from a survey of college undergraduates by
Ward and Ault (1990) (student2.dta).
.

describe

Contains data from C:\data\student2.dta
obs:
243
vars:
19
size:
6,561 (99.9% of memory free)
variable name

j

*&a

id
year
age
gender
major
relig
drink
gpa
grades

storage
type

int
byte
byte
byte
byte
byte
byte
float
byte

display
format
%8.0g
%8.0g
%8.0g
%9.0g
%8.0g
%8.0g
%9.0g
%9.0g
% 8 . Og

value
label

Student survey (Ward & Ault 1990)
12 Jul 2005 10:16

variable label

year

Student ID
Year in college
Age at last birthday

s

Gender

v4

grades-

(male)

Student major
Religious preference
33-point drinking scale
Grade Point Average
Gtressed-grades this semester

1

ANOVA and Other Comparison Methods

belong
live
miles
study
athlete
employed
allnight
ditch
hsdrink
aggress

byte
byte
byte
byte
byte
byte
byte
byte
byte
byte

Sorted by:

%8.0g
%8.0g
%8.0g
%8.0g
%8.0g
%8.0g
%8- 0g
%8.0g
%9.0g
%9.0g

belong
vlO

147

Belong to fraternity/sorority
Where do you live?
How many miles from campus?
Avg. hours/week studying
Are you a varsity athlete?
Are you employed?
How often study all night?
How many class/month ditched?
High school drinking scale
Aggressive behavior scale

yes
yes
allnight
times

id

About 19% of these students belong to a fraternity or sorority:
.

I

tabulate belong

Belong to |
fraternity/ I
sorority I
----------- +
member I
nonmember |

Freq.

Percent

Cum.

47
196

19.34
80.66

19.34
100.00

Total |

243

100.00

Another variable, drink, measures how often and heavily a student drinks alcohol, on a 33point scale. Campus rumors might lead one to suspect that fraternity/sorority members tend to
differ from other students in their drinking behavior. Box plots comparing the median drink
values of members and nonmembers, and a bar chart comparing their means, both appear
consistent with these rumors. Figure 5.1 combines these two separate plot types in one image.
.

graph box drink,

ylabel(0(5)35)

over (belong)

saving(fig05_01a)

. graph bar (mean) drink, over(belong) ylabel(0(5)35) saving(fig05_01b)

.

graph combine fig05_01a.gph fig05_01b .gph,

col(2)

iscale(1.05)

Figure 5.1
m _
co

co

o

CO

o
co ’

gCN -

CN

m _

(/)

co _

"O O _

V? CM

c
"O L0

-

E

9-o

Or- "
CO

o

■pg

in

o

member
i

1

nonmember

o
member

nonmember

148

Statistics with Stata

^eSt icommand’ use<i earlier for one-sample and paired-difference tests, can
perform two-sample tests as well. IIn this application its general syntax is ttest
measurement, by (categorical). For example,
.

ttest drink, by(belong)

Two-sample t test with equal variances
Group |
-------- + —
member |
nonmembe |

Obs

Mean

Std. Err-.

Std. Dev.

[95% Conf. Interval]

47
196

24.7234
17.7602

.7124518
.4575013

.4.884323
*6.405018

23.28931
16.85792

26.1575
18.66249

combined |

243

19.107

.431224

6.722117

18.25756

19.95643

6.9632

.9978608

4.997558

8.928842

diff |
Degrees of freedom: 241

Ho: mean(member) - mean(nonmembe) = diff = 0

Ha: diff < 0
t =
6.9781
P < t =
1.0000

.

L4

Ha: diff > 0
t =
6.9781
P > t =
0.0000

ttest drink, by(belong) unequal

Two-sample t test with unequal variances
Group |

Obs

Mean

Std. Err.

Std. Dev.

[95% Conf. Interval]

47
196

24.7234
17.7602

.7124518
.4575013

4.884323
6.405018

23.28931
16.85792

26.1575
18.66249

243

19.107

.431224

6.722117

18.25756

19.95643

6.9632

. 8466965

5.280627

8.645773

I---

■i

Ha: diff != 0
t =
6.9781
P > It| =
0.0000

member |
nonmembe |
-------- +--combined |
-------- +--diff |

Satterthwaite's degrees of freedom:

88.22

Ho: mean(member) - mean(nonmembe)
= diff = 0
Ha: diff < 0
Ha: diff •= 0
Ha: diff > 0
t = 8.2240
t =
8.2240
t =
8.2240
P < t =
1.0000
P > It| =
0.0000
P > t =
0.0000
‘k

le^ank disti ibutions have similar shape, the rank-sum test here indicates that we can reject the
null hypothesis of equal population medians.

4




ANOVA and Other Comparison Methods

.

ranksum drink,

by(belong)

Two-sample Wilcoxon rank-sum

I

(Mann-Ahltney)

tesr

belong I

cbs

member |
nonmember i

47
1 96

853 5
21111

5734
23912

combined I

243

2 9 64c

29646

rank

unadjusted variance
adjustment for ties

187310.67
-472.30

adjusted variance

186838.36

Ho: drink(belong
z =
Prob
IZ| =

149

member)
6.480
0.0000

expected

dr in k(be long

nonmer.ber)

One-Way Analysis of Variance (ANOVA)

Analysis »f ..ri.nee (ANOVA) provides anoiher way. more general rhan, tests to test (or
differences among means The simplest case, one-way ANOVA. tests whether the means of
differ across categories of,. One-way ANOVA can be performed bv a on.w.y command
with the general form oneway measurement categorical. Forexample.
.

I

oneway drink belong,

Belong to I
fraternity/ |
sorority |

member |
nonmember I
Total |

tabulate

Summary of 33-point dtinkina s~aiMean
Std. Dev.
'
n a

60204
X

4.8843233
6.4 0 51 ' -

• A

= .':ce
MS

Source
Between groups
Within groups
Total

1838.08426
9097.13385

241

1838.08426
37.7474433

2131242

4r.18685:"

Bartlett's test for equal variances:

chi. (1)

=

r

Prob > F

4 8.69

0.1000

4.8378

9rob>chi2 = 0.128

abandoning the equal-variances assumption.
'
K
Ba d "tr37 Jhrnally TS ‘i6 equa|-variances assumption, using Bartlett’s X2.
. A low
Bartlett s probability implies that ANOVA’s c----- ’ ’
equal-variance assumption is implausible, in

i

J

150
.i. J

Statistics with Stata

which case we should not trust the ANOVA F test results. In the coneway drink
. - - belong
- example above, Bartlett’s P = .028 casts doubt on the ANOVA’s validity.
ANOVA s real value lies not in two-sample comparisons, but in more complicated
comparisons of three or more means. For example, we could test whether mean drinking
behavior varies by year in college:
"

fed
ini
will

.

J

oneway drink year,

Year in | Summary of 33-point drinking scale
Mean
Std. Dev.
Freq.

college
““1’ ’ 3
I
Freshman
Sophomore
Junior
Senior

|
I
|
|

18.975
21.169231
19.453333
16.650794

6.9226033
6.5444853
6.28660S1
6.6409257

‘40

Total |

19.106996

6.7221166

243

'a

I

Between groups
Within groups

666.200518
10269.0176

3
239

222.066839
42.9666038

Total

10935.2181

242

45.186851“

Bartlett' s test for equal variances:

B;

Row Mean-|
Col Mean |

-------------- +

4 !

65
75
63

Analysis cf Variance
SS
df
MS

Source

Sophomor

|
I
I
Junior I
I
I
Senior I
I

chi2(3) =

F

Prob > F

5.17

0.0018

0.5103

Prob>chi2 = 0.917

Comparison of 33-point drinking scale by Year in college
(Scheffe)
Freshman

Sophomor

Junior

2.19423
0.429

.478333
0.987

-1.7159
0.498

-2.32421
0.382

-4.51844
0.002

-2.83254

We can reject the hypothesis of equal means (P = .0018). but not the hypothesis of equal
variances (P - .917). The latter is “good news" regarding the ANOVA’s validity.

Pl°ts ’n Figure 5.2 (next page) support this conclusion, showing similar variation
within each category. This figure, which combines separate box plots and dot plots, shows that
differences among medians and among means follow similar patterns.

1»•

J i!

I
■'

tabulate scheffe



saving (,fig05_02a)

.

graph hbox drink, over (year) ylabel (0 (5) 35)

.

graph dot (mean) drink, over(year) ylabel (0 (5)35, grid)
marker (1, msymbol(S)) saving(fig05_02b)

.

graph combine fig05_02a . gph fig05__02b. gph ,

row(2)

iscale(1.05)

•W
ANOVA and Other Comparison Methods

151

Figure 5.2

Freshman
Sophomore

Junior

Senior
0

5

10
15
20
25
33-point drinking scale

30

35

30

35

Freshman
Sophomore
Junior
Senior

0

10

5

15
20
mean of drink

25

The scheffe option (Scheffe multiple-comparison test) produces a table showing the
differences between each pair ofmeans. The freshman mean equals 18.975 and the sophomore
mean equals 21.16923, so the sophomore-freshman difference is 21.16923 - 18 975 = 2 19423
not statistically distinguishable from zero (P = .429). Of the six contrasts in this table only the
senior-sophomore difference, 16.6508 -21.1692 =-4.5184, is significant (P= .002). Thus,
our overall conclusion that these four groups’ means are not the same arises mainly from the
contrast between seniors (the lightest drinkers) and sophomores (the heaviest).
oneway offers three multiple-comparison options: scheffe , bonferroni , and
sidak (see Base Reference Manual for definitions). The Scheffe test remains valid under
a wider variety of conditions, although it is sometimes less sensitive.
The Kruskal-Wallis test ( kwallis ), a A'-sampIe generalization of the two-sample rank­
sum test, provides a nonparametric alternative to one-way ANOVA. Ittests the null hypothesis
of equal population medians.
.

kwallis drink,

Test:

Equality of populations (Kruskal-Wallis test)
year

I

I

by(year)

I
Freshr.ar. i
I Sophomore I
I
Junior |
I
Senior I

chi-squared =
probability =

Obs

I Rank Sum |
---- I
40 |
4914.00 |
65 |
9341.50 |

75
63

|
|

9300.50
6090.00

|
|

14.453 with 3 d.f.
0.0023

chi-squared with ties =
probability =
0.0023

14.490 with 3 d. f.

Gs- ICrO
1U002

.0

<C

.

) *
) -

152

*

I

Statistics with Stata

Here, the kwallis results (P= .0023) agree with our oneway findings of significant
differences in drink by year in college. Kruskal-Wallis is generally safer than ANOVA if we
have reason to doubt ANOVA’s equal-variances or normality assumptions, or if we suspect
problems caused by outliers, kwallis , like ranksum. makes the weaker assumption of
similar-shaped distributions within.each group. In principle, ranksum and kwallis
should produce similar results when applied to two-sample comparisons, but in practice this is
true only if the data contain no ties, ranksum incorporates an exact method for dealing with
ties, which makes it preferable for two-sample problems.

Two- and A/-Way Analysis of Varianee
One-way ANOVA examines how the means of measurement variable;/ vary across categories
of one other variable x. N-way ANOVA generalizes this approach to deal with two or more
categorical x variables. For example, we might consider how drinking behavior varies not only
by fiaternity or soiority membership, but also by gender. We start by examining a two-way
table of means:

I

.

S'-

i


4 I

table belong gender,

Belong to |
fraternit I
y/sororit I
Y
I

----------------+.

Gender (male)
Female
Male

member I 22.44444
nonmember | 16.51724

i

fl
111
l-H

I

??h

r!

contents(mean drink)

row col

Total

26.13793
19.5625

24.7234
17.7602

21.31193

19.107

I
Total

I

17.31343

It appears that in this sample, males drink more than females and members drink more than
nonmembers. The member-nonmember difference appears similar among males and females.
Stata’s 2V-way ANOVA command, anova , can test for significant differences among these
means attributable to belonging to a fraternity or sorority, gender, or the interaction of
belonging and gender (written belong*gender).
anova drink belong gender belong* gender
Number of obs =
Root MSE

R-squared
Adj R-squared =

C.2222
0.2123

Source

Partial SS

dr

MS

F

Prob > r

Mode 1

242 5.-'237

3

309.557456

22.75

0.0000

1416.2366
408.520097
3.7=216612

1
1

1416.2366
403.520097
3 . " = 016612

39.51
11 . 48
0.11

0.0000
0.0008
0.7448

I
I
belong |
gender |
belong*gender |

I
Residual

I

8506.54574

239

35.5922416

Total

|

10935.2181

242

4 5 . 1868517

ANOVA and Other Comparison Methods

153

In this example of“two-way factorial ANOVA,” the output shows significant main effects
for belong^(P = .0000) and gender (P = .0008), but their interaction contributes little to the
model (P - .7448). This interaction cannot be distinguished from zero, so we might prefer to
fit a simpler model without the interaction term (results not shown):
anova drink belong gender

To include any interaction term with anova , specify the variable names joined by *.
Unless the number of observations with each combination of x values is the same (a condition
called "balanced data”), it can be hard to interpret the main effects in a model that also includes
inteiactions. This does not mean that the main effects in such models are unimportant,
however. Regression analysis might help to make sense of complicated ANOVA results, as
illustrated in the following section.

Analysis of Covariance (ANCOVA)
Analysis of Covariance (ANCOVA) extends V-way ANO VA to encompass a mix ofcategorical
and continuous^ variables. This is accomplished through the anova command if we specify
which variables are continuous. For example, when we include gpa (college grade point
average) among the independent variables, we find that it, too, is related to drinking behavior.

I
I

anova drink belong gender gpa, continuous(gpa)

Number of obs =
218
Root MSE
= 5.68939

I

R-squared
=
Adj R-squared =

0.2970
0.2'8 72

Source |

Partial SS

df

MS

F

Prob > F

Model |
I
belong |
gender |
gpa I
I
Residual I

2927.03087

3

975.676958

30.14

0.0000

1489.31999
405.137843
407.0089

1
1
1

1489.31999
405.137843
407.0089

46.01
12.52
12.57

0.0000
0.0005
0.0005

6926.99206

214

32.3691218

Total |

9854.02294

217

45.4102439

From this analysis we know that a significant relationship exists between drink and gpa
when we control for belong and gender. Beyond their F tests for statistical significance,
however, ANOVA or ANCOVA ordinarily do not provide much descriptive information about
how variables aie related. Regression, with its explicit model and parameter estimates, does
a better descriptive job. Because ANOVA and ANCOVA amount to special cases of
regression, we could restate these analyses in regression form. Stata does so automatically if
we add the regress option to anova. For instance, we might want to see regression
output in order to understand results from the following ANCOVA.

I

*1

154

Statistics with Stata

anova
regress

Source |

----- +

belong gender belong*gender gpa,

SS

df

MS

Model I
Residual |

2933.45823
6920.5647

4.
213

733.364558
32.4909141

Total

9854.02294

217

45.4102439

drink

continuous(gpa)

Number of obs =
F ( 4,
213) =
Prob > F
=
R-squared
=
Adj R-squared =
Root MSE
=

218
22.57
0.0000
0.2977
0.2845
5.7001

Coef.

Sid. Err.

t

P> I t I

[95% Conf

Interval]

27.47676

2.439962

11.26

0.000

22.6672

32.28633

1
2

6. 925384
(dropped)

’ .286774

5.38

0.000

4.388942

9.461826

1
2

-2.629057
(dropped)
-3.054633

. =917152

-2.95

0.004

-4.386774

-.8713407

. =5934 98

-3.55

0.000

-4.748552

-1.360713

1.946211

-0.44

0.657

-4.701916

2.970685

_cons
belong

gender

gpa
belong*gender
1
1
2
2

1
2
1
2

-.8656158
(dropped)
(dropped)
(dropped)

With the jregress option, we get the anova output formatted as a regression table.
The top part gives the same overall F
andas
R a standard ANOVA table. The bottom part
- test
-------describes the following regression:
We construct a separate dummy variable {0,1} representing each category of each x
variable, except for the highest categories, which are dropped. Interaction terms (if
specified in the variable list) are constructed from the products of every possible
combination of these dummy variables. Regressy on all these dummy variables and
interactions, and also on any continuous variables specified in the command line.
The previous example therefore corresponds to a regression of drink on four x variables:
1. a dummy coded 1 = fratemity/sorority member, 0 otherwise (highest category of belong
nonmember, gets dropped);
2. a dummy coded 1 = female, 0 otherwise (highest category ofgender, male, gets dropped);

3.
4.

the continuous variable gpa;

an interaction term coded 1 = sorority female, 0 otherwise.
Interpret the individual dummy variables’ regression coefficients as effects on predicted or
"a,ean?’; ForexamPle> the coefficient on the first category ofgender (female) equals-2.629057
I his informs us that the mean drinking scale levels for females are about 2.63 points lower than
those of males with the same grade point average and membership status. And we know that
among students of the same gender and membership status, mean drinking scale values decline
y . 54633 with each one-point increase in grades. Note also that we have confidence
interva s and individual t tests for each coefficient; there is much more information in the
anova, regress output than in the ANOVA table alone.

I
i,

J

ANOVA and Other Comparison Methods

155

Predicted Values and Error-Bar Charts

After anova, the followup command predict calculates predicted values, residuals or
standard errors and diagnosttc statistics. One application for such statistics is in drawing
graphical representat.ons of the model’s predictions, in the form of error-bar charts For Z
simple illustration, we return to the one-way ANOVA of drink by year.
anova drink year

Number of obs =
243
Root MSE
= 6.55489

R-squared
Adj R-squared =

0.0609
0.0491

Source |

Partial SS

df

MS

F

Prob > F

Model

666.200518

3

222.066839

5.17

0.0018

5.17

0.0018

I
i
year l
i
Residual i

666.200518

3

222.066839

10269.0176

239

42.9666008

Total

10935.2181

242

45.1868517

|

To calculate predicted means from the recent anova,type predict followed by a new
variable name:
. predict drinkmean
(option xb assumed; fitted values)
.

label variable drinkmean "Mean drinking scale"

With the Stdp option, predict calculates standard errors of the predicted means:
.

predict SEdrink,

stdp

Using these new variables, we apply the serrbar command to create an error-bar chart
The scale (2) option tells serrbar to draw error bars of plus and minus two standard
errors, from
drinkmean - 2*SEdrink
to
drinkmean + 2*SEdrink.
In a serrbar command, the first-listed variable should be the means or y variable* the
second-listed the standard error or standard deviation (depending on which you want to show);
and the third-listed variable defines the x axis. The plot ( ) option for serrbar can
specify a second plot to overlay on the standard-error bars. In Figure 5.3, we overlay a line plot
that connects the drinkmean values with solid line segments.

156

Statistics with Stata

serrbar drinkmean SEdrink year,

.

plot (line drinkmean year,

scale(2)

cipattern(solid))

legend(off)
Figure 5.3

CM
CM ’

rLi

1

■Q

1
CD

1

2
Year in college

I
4 I

3

4

For a two-way factorial ANOVA, error-bar charts help us to visualize main and interaction
effects. Although the usual error-bar command serrbar can, with effort, be adapted for this
purpose, an alternative approach using the more flexible graph twoway family will be
illustrated below. First, we perform ANOVA, obtain group means (predicted values) and their
standard errors, then generate new variables equal to the group means plus or minus two
standard errors. The example examines the relationship between students’ aggressive behavior
(aggress), gender, and year in college. Both the main effects of gender Midyear, and their
interaction, are statistically significant.
anova aggress gender year gender*year

Number of obs =
243
Root MSE
= 1.45652
Source |

Partial SS

df

Model

R-squared
=
Adj R-squared =

MS

0.2503
0.2280

F

Prob > F

I
I
gender |
year I
gender*year |

166.432503

7

23.7832147

11.21

0.0000

94.3505972
1 9.0404045
24.1029759

1
3
3

44 . 47
2.99
3.79

0.0000
0.0317
0.0111

Residual

I
I

94.3505972
6.34680149
8.03432529

498.538073

235

2.12143861

Total

|

665.020576

242

2.74801891

ANOVA and Other Comparison Methods

157

. predict aggmean

(option xb assumed; fitted values)
$
■w..

. label variable aggmean "Mean aggressive behavior scale"
. predict SEagg, stdp
gen agghigh - aggmean + 2 * SEagg

.

. gen agglow — aggmean - 2 * SEagg

. graph twoway connected aggmean year

I I
I I

reap agghigh agglow year

/ by(gender, legend(off) note(
))
ytitle("Mean aggressive behavior scale")
Female

Figure 5.4

Male

CO

.0)

o
w
o
(N -

<D

n
§
w
V)

s?O)

*

D)

ro
c

IQ
0)

2

o

3

4 1
Year in college

2

3

4

Figure 5.4 built error-bar charts by overlaying two pairs of plots. The first pair are female
and male connected-line plots, connecting the group means of aggress (which we calculated
using predict, and saved as the variable aggmean). The second pair are female and male
capped-spike range plots (twoway reap) in which the vertical spikes connectine variables
agghigh (group means of aggress plus two standard errors) and agglow (group'”means of
aggress minus two standard errors). The by (gender) option produced sub-plots for females
and males. Notice that to suppress legends and notes in a graph that uses a by ( ) option,
legend (off) and note ("") must appear as suboptions within by ( ).

The resulting error-bar chart (Figure 5.4) shows female means on the aggressive-behavior
scale fluctuating at comparatively low levels during the four years of college. Male means are
higher throughout, with a sophomore-year peak that resembles the pattern seen earlier for
rinkmg (Figures 5.2 and 5.3). Thus, the relationship between aggress and year is different for
males and females. This graph helps us to understand and explain the significant interaction
eiieci.

predict works the same way with regression analysis ( regress ) as it does with
anova because the two share a.cpmmon mathematical framework. A list of some other

i!I'

Ifel
I'

158

Statistics with Stata

,°PtIOnS aPPearS in Chapter 6’ and filrther examPles usi"gthese oP^ons are given
m Chapter 7. The options include residuals that can be used to check assumptions regarding
X™^OnS’
alS° H SUite °f dia8n0Stic statistics <such as leverage. Cook’s D, and
n u
1 meaSUre thC inflUenCe °f
observations on model results The
Ourbm Watson test ( dwstat), described in Chapter 13, can also be used after anova to
test for fet-order autocorrelation. Conditional effect plotting (Chapter 7) provides a graphical
approac that can aid interpretation of more complicated regression, ANOVA or ANCOVA
models.


Linear Regression Anaiysis

Stata offers an exceptionally broad range of regression procedures. A partial list of the
possibilities can be seen by typing help ’ regress . This chapter introduces regress
and related commands that perform simple and multiple ordinary least squares (OLS)
regression. One followup command, predict, calculates predicted values, residuals, and
diagnostic statistics such as leverage or Cook’s D. Another followup command, test
performs tests of user-specified hypotheses, regress can accomplish other analyses
including weighted least squares and two-stage least squares. Regression with dummy
variables, interaction effects, polynomial terms, and stepwise variable selection are covered
briefly in this chapter, along with a first look at residual analysis.

The following menus access most of the operations discussed:
Statistics - Linear regression and related - Linear regression
Statistics - Linear regression and related - Regression diagnostics
Statistics - General post-estimation - Obtain predictions, residuals, etc., after estimation
Graphics - Overlaid twoway graphs

Statistics - Cross-sectional time series

Example Commands
. regress y x

Performs ordinary least squares (OLS) regression of variable v on one predictor, x.
. regress y x if ethnic == 3 & income > 50

Regresses y on x using only that subset of the data for which variable ethnic equals 3 and
income is greater than 50.
. predict yhat

Generates a new variable (here arbitrarily namedyhat) equal to the predicted values from
the most recent regression.
. predict e, resid

Geneiates a new variable (here arbitrarily named e) equal to the residuals from the most
recent regression.
. graph twoway Ifit y x

||

scatter y x

Draws the simple regression line (Ifit or linear fit) with a scatterplot ofy vs. x.

159

160

I

Statistics with Stata

. graph twoway mspline yhat x

||

scatter y x

Draws a simple regression line with a scatterplot ofy vs. x by connecting (with a smooth
cubic spline curve) the regression’s predicted values (in this example named vhat).
Note: There are many alternative ways to draw regression lines or curves in Stata.
These alternatives include the. twoway graph types mspline (illustrated above),
mband, line , If it, If itci , qf it, and qf itci , each of which has its own
advantages and options. Usually we combine (overlay) the regression line or curve with
a scatterplot. If the scatterplot comes second in our graph twoway command, as in
the example above, then scatterplot points will print on top of the regression line. Placing
the scatterplot first in the command causes the line to print on top of the scatter. Examples
throughout this and the following chapters illustrate some of these different possibilities.

I
r

I

J
a

.

1 J'

rvfplot

Draws a residual versus fitted (predicted values) plot, automatically based on the most
recent regression.

f!

. graph twoway scatter e yhat, yline(O)

Draws a residual versus predicted values plot using the variables e and yhat.
.

i

regress y xl x2 x3

Performs multiple regression ofy on three predictor variables, x/, x2, and x3.
.

regress y xl x2 x3, robust

Calculates robust (Huber/White) estimates of standard errors. See the User's Guide for
details. The robust option works with many other model fitting commands as well.
. regress y xl x2 x3, beta

I
5

F

Performs multiple regression and includes standardized regression coefficients (“beta
weights”) in the output table.
. correlate xl x2 x3 y

Displays a matrix of Pearson correlations, using only observations with no missing values
on all of the variables specified. Adding the option covariance produces a variance­
covariance matrix instead of correlations.
. pwcorr xl x2 x3 y, sig

Displays a matrix of Pearson correlations, using pairwise deletion of missing values and
showing probabilities from t tests of H0:p = 0 on each correlation.
. graph matrix xl x2 x3 y, half

Draws a scatterplot matrix. Because their variable lists are the same, this example yields
a scatterplot matrix having the same organization as the correlation matrix produced by the
preceding pwcorr command. Listing the dependent (y) variable last creates a matrix in
which the bottom row forms a series ofj>-versus-x plots.
test xl x2

Performs an Ftest of the null hypothesis that coefficients onx7 andx? both equal zero in
the most recent regression model.
.xi:

regress y xl x2 i.catvar*x2

Performs “expanded interaction” regression ofy on predictors x/, x2, a set of dummy
variables created automatically to represent categories of catvar, and a set of interaction
terms equal to those dummy variables times measurement variable x2. help xi gives
more details.
..
.
.
.

Linear Regression Analysis

.

161

sw regress y xl x2 x3, pr(.O5)

Performs stepwise regression using backward elimination until all remaining predictors are
significant at the .05 level. All listed predictors are entered on the first iteration.
Thereafter, each iteration drops one predictor with the highest P value, until all predictors
remaining have probabilities below the “probability to retain,” pr (. 05). Options permit
forward or hierarchical selection. Stepwise variants exist for many other model-fitting
commands as well; type help sw for a list.
. regress y xl x2 x3 [aweight = w]

Perfonns weighted least squares (WLS) regression ofy onxl,x2, andx3. Variable w holds
the analytical weights, which work as if we had multiplied each variable and the constant
by the square root of w, and then performed an ordinary regression. Analytical weights are
often employed to correct for heteroskedasticity when they and x variables are means,
rates, or proportions, and w is the number of individuals making up each aggregate
observation (e.g., city or school) in the data. If they and x variables are individual-level,
and the weights indicate numbers of replicated observations, then use frequency weights
[fweight = v] instead. See help svy if the weights reflect design factors such
as disproportionate sampling.
. regress yl y2 x (x z)
. regress y2 yl z (x z)

Estimates the reciprocal effects ofy/ and v2. using instrumental variables x and z. The
first parts of these commands specify the structural equations:
yl = a0 + a}y2 + a2x + e,
v2 = po + P|^2 + p2M'4-e2
The parentheses in the commands enclose variables that are exogenous to all of the
structural equations, regress accomplishes two-stage least squares (2SLS) in this
example.
.

svy:

regress y xl x2 x3

Regresses von predictors.v/..v2. and.v3, with appropriate adjustments fora complex survey
sampling design. \\ e assume that a svyset command has previously been used to set
up the data, by specifying the strata, clusters, and sampling probabilities, help svy
lists the many procedures available for working with complex survey data, help
regress outlines the syntax of this particular command; follow references to the User's
Guide and the Surxfey Data Reference Manual for details.
. xtreg y xl x2 x3 x4, re

Fits a panel (cross-sectional time series) model with random effects by generalized least
squares (GLS). An observation in panel data consists of information about unit i at time
and there are multiple observations (times) for each unit. Before using xtreg , the
variable identifying the units was specified by an iis (“z is”) command, and the variable
identifying time by tis (‘7 is”). Once the data have been saved, these definitions are
retained for future analysis by xtreg and other xt procedures, help xt lists
available panel estimation procedures, help xtreg gives the syntax of this command
and references to the printed documentation. If your data include many observations for
each unit, a time-series approach could be more appropriate. Stata’s time series procedures
(introduced in Chapter 13) provide further tools for analyzing panel data. Consult the
Longitudinal/Panel Data Reference Manual for a full description.
I

162

.

1

Statistics with Stata

xtmixed population year

city: year

||

Assume that we have yearly data on population, for a number of different cities. The
xtmixed population year part specifies a“fixed-effect”model,similarto ordinary
regression, which describes the average trend in population. The || city: year part
specifies a “random-effects” model, allowing unique intercepts and slopes (different
starting points and growth rates) for each city.
. xtmixed SAT grades prepcourse

||

district:

pctcollege

||

region:

Fits a hierarchical (nested or multi-level) linear model predicting students’s SAT scores as
a function of the individual students’ grades and whether they took a preparation coursethe percent college graduates among their school district’s adults; and region of the country
(region affecting v-intercept only). See the Longitudinal/Panel Data Reference Manual for
much more about the xtmixed command, which is new with Stata 9.

The Regression Table
File states.dta contains educational data on the U.S. states and District of Columbia:
describe state csat expense percent income high college region
variable name

state
csat
expense
percent
income
high
college
region

storage
type

display
format

str20
int
int
byte
long
float

%20s
%9.0g
%9.0g
%9.0g
*10.0g
%9.0q'
-9.0g
9.0g

byte

value
label

variable label

region

State
Mean composite SAT score
Per pupil expenditures prim&sec
% HS graduates taking SAT
Median household income
=: adults HS diploma
adults college degree
Geographical region

Political leaders occasionally use mean Scholastic Aptitude Test (SAT) scores to make
pointed comparisons between the educational systems of different U.S. states. For example,
some have raised the question ot whether SAT scores are higher in states that spend more
money on education. We might try to address this question by regressing mean composite SAT
scores (csat) on per-pupil expenditures (expense). The appropriate Stata command has the form
regress y x , where y is the predicted or dependent variable, and x the predictor or
independent variable.
regress csat expense

Source !

ss

df

MS

Number of obs =
F( 1,
49) =
Prob > F
=
R-squared
=
Adj R-squared =
Root MSE

Model |
Residual |

48708.3001
175306.21

1
49

48708.3001
3577.67775

Total |

224014.51

50

4480.2902

csat I

Coe f.

Std. Err.

expense |
_cons I

- . 0222756
1060.732

. 0060371

-3.69

0.001

.. 32._7Q 0.9

32.

o.uiao—

t

P> 111

51
13.61
0.0006
0.2174
0.2015
59.814

[95% Conf. Interval]

-.0344077
995.0175

-.0101436
1126.447

I

5
Linear Regression Analysis

163

This regression tells an unexpected story: the more money a state spends on education, the
lower its students’ mean SAT scores. Any causal interpretation is premature at this point,’but
the regression table does convey information about the linear statistical relationship between
csat and expense. At upper right it gives an overall Atest, based on the sums of squares at the
upper left. This F test evaluates the null hypothesis that coefficients on all x variables in the
model (here there is only onex variable, expense) equal zero. The/7statistic, 13.61 with I and
49 degrees of freedom, leads easily to rejection of this null hypothesis (P = .0006). Prob > F
means "the probability ofa greater/7” statistic if we drew samples randomly from a population
in which the null hypothesis is true.

At upper right, we also see the coefficient of determination, R 2 = .2174. Per-pupil
expenditures explain about 22% of the variance in states’ mean composite SAT scores.
Adjusted R , R a = .2015, takes into account the complexity of the model relative to the
complexity of the data. This adjusted statistic is often more informative for research.
The lower half of the regression table gives the fitted model itself. We find coefficients
(slope and r-intercept) in the first column, here yielding the prediction equation
predicted csat = 1060.732 - .0222756expense

The second column lists estimated standard errors of the coefficients. These are used to
calculate t tests (columns 2-4) and confidence intervals (columns 5-6) for each regression
coefficient. The t statistics (coefficients divided by their standard errors) test null hypotheses
that the corresponding population coefficients equal zero. At the a = .05 significance level, we
could reject this null hypothesis regarding both the coefficient on expense (P = .001) and the
j-intercept ( .000 , really meaning P < .0005). Stata’s modeling commands print 95%
confidence intervals routinely, but we can request other levels by specifying the level ( )
option, as shown in the following:
. regress csat expense, level(99)

Because these data do not represent a random sample from some larger population of U.S.
states, hypothesis tests and confidence intervals lack their usual meanings. They are discussed
in this chapter anyway for purposes of illustration.
The term cons stands tor the regression constant, usually set at one. Stata automatically
includes a constant unless we tell it not to. The nocons option causes Stata to suppress the
constant, performing regression through the origin. For example,
.

regress y x,

nocons

or

.

regress y xl x2 x3, nocons

In certain adxanced applications, you might need to specify your own constant. If the
independent variables include a user-supplied constant (named c, for example), employ the
hascons option instead of nocons :
.

regress y c x, hascons

Using nocons in this situation would result in a misleading/7 test and/?2. Consult the Base
Reference Manual or help regress for more about hascons .

164

Statistics with Stata

Multiple Regression
r

Multiple regression allows us to estimate how expense predicts csat, while adjusting for a
number of other possible predictor variables. We can incorporate other predictors of csat
simply by listing these variables in the command
.

regress

csat expense percent income high college

Source

SS

df

MS

Model
Residual

I
I

184663.309
39351.2012

45

36932.6617
=74.471137

Total

|

224014.51

50

4480.2902

csat |

------- + _
expense
percent
income
high
college
_cons

I
|
I
|
I
|

Coe f .
. 0033528
-2.618177
.0001056
1.630841
2.030894
851.5649

Std .

. 0044"09
.2536491
.0011661
.992247
1.660128
5 9.29228

Number of obs =
F( 5,
45) =
Prob > F
R-squared
Adj R-squared =
Root MSE
?> I c

0.75
-10.31
0.09
1.64
1.22
14.36

o. o :
J. 92 =
O.IC"
3.22 =
:. c: i

51

42.23
0.0000
0.8243
0.8048
29.571

[95% Conf

Interval]

-.005652
-3.129455
-.002243
-.367647
-1.312756
732.1441

.0123576
-2.106898
.0024542
3.629329
5.374544
970.9857

fit I
This yields the multiple regression equation

predicted csat - 851.56 + .00335eApense - 2.61 ^percent + .OOQMnconie +
1.62>high -f- Z.Qlcollege

Controlling for four other variables weakens the coefficient on expense from -.0223 to .00335,
which is no longer statistically distinguishable fromzero. The unexpected negative relationship
between expense and csat found in our earlier simple regression evidently can be explained by
other predictors.

r

J




it

Only the coefficient on percent (percentage of high school graduates taking the SAT)
attains significance at the .05 level. We could interpret this “fourth-order partial regression
coefficient” (so called because its calculation adjusts for four other predictors) as follows.

^2
2.618. Predicted mean SAT scores decline by 2.618 points, with each one-point
increase in the percentage of high school graduates taking the SAT — if expense,
income, high, and college do not change.
Taken together, the five .v variables in this model explain about 80% of the variance in
states mean composite SAT scores (R= .8048). In contrast, our earlier simple regression
with expense as the only predictor explained only 20% of the variance in csat.
To obtain standardized regression coefficients (“beta weights”) with any regression, add
the beta option. Standardized coefficients are what we would see in a regression where all
the variables had been transformed into standard scores (means 0, standard deviations 1).

Linear Regression Analysis

.

165

regress csat expense percent Income high college, beta

Source |

ss

df

MS

Model |
Residual |

184663.309
39351.2012

5
45

36932.6617
874.471137

Total |

224014.51

50

4480.2902

csat |

Coef.

Std. Err.

t

P>l 11

Beta

.0033528
-2.618177
.0001056
1.630841
2.030894
851.5649

.0044709
.2538491
.0011661
.992247
1.660118
59.29228

0.75
-10.31
0.09
1.64
1.22
14.36

0.457
0.000
0.928
0.107
0.228
0.000

.070185
-1.024538
.0101321
.1361672
. 1263952

expense
percent
income
high
college
_cons

I
|
|
|
|
|

Number of obs =
F( 5,
45) =
Prob > F
R-squared
Adj R-squared =
Root MSE

51
42.23
0.0000
0.8243
0.8048
29.571

The standardized regression equation is
predicted csat* = .07expense* - \XftA5percent* + .Oiinconie* +
A36high* + .126co//ege*
where csat*, expense*, etc. denote these variables in standard-score form. We might interpret
the standardized coefficient on percent, for example, as follows:
^2
1.0245. Predicted mean SAT scores decline by 1.0245 standard deviations,
with each one-standard-deviation increase in the percentage of high school graduates
taking the SAT — if expense, income, high, and college do not change.

The F and t tests, R2, and other aspects of the regression remain the same.

Predicted Values and Residuals
After any regression, the predict command can obtain predicted values, residuals, and
other case statistics. Suppose we have just done a regression of composite SAT scores on their
strongest single predictor:
. regress csat percent

Now, to create a new variable called yhat containing predicted v values from this regression,
type
. predict yhat

.

label variable yhat "Predicted mean SAT score"

Through the resid option, we can also create another new variable containing the
residuals, here named e:
. predict e,

resid
label variable e "Residual"

We might instead have obtained the same predicted y and residuals through two
generate commands:
.

generate yhatO — _b[_cons] + __b [percen t ] ★percen t

I

p

166

n

Statistics with Stata

Sir
generate eO = csat - yhatO

.

Stata temporarily remembers coefficients and other details from the recent regression. Thus
_b[vfl/ name ] refers to the coefficient on independent vanable varname. b[ cons] refers to
the coefficient on _cons (usually, the y-intercept). These stored values are useful in
programming and some advanced applications, but for most purposes, predict saves us the
trouble of generating^yhatO and eO “by hand” in this fashion.
Residuals contain information about where the model fits poorly, and so are important for
diagnostic or troubleshooting analysis. Such analysis might begin just by sorting and
examining the residuals. Negative residuals occur when our model overpredicts the observed
values. That is, in these states the mean SAT scores are lower than we would expect, based on
what percentage of students took the test. To list the states with the five lowest residuals, type

I

sort e
.

list state percent csat yhat e in 1/5

+
i
i
i
2
3
4
5

!
I

state

percent

csat

yhat

South Carolina
West Virginia
North Carolina
Texas
Nevada

58
17
57
44
25

832
926
844
874
919

894.3333
986.0953
896.5714
925.6666
968.1905

e I
-62.3333
-60.09526
-52.5714
-51.66666
-49.19049

|
|
|
|
|

The four lowest residuals belong to southern states, suggesting that we might be able to improve
our model, or better understand variation in mean SAT scores, by somehow taking region into
account.
Positive residuals occur when actual y values are higher than predicted. Because the data
already have been sorted by e, to list the five highest residuals we add the qualifier
‘•-5" in this qualifier means the 5th-from-last observation, and the letter “el” (note that this is
not the number “1”) stands for the last observations. The qualifiers in 47/1 or in
47/51 could accomplish the same thing.
.

I

list state percent csat yhat e in -5/1

47 .
48 .
49.
50 .
51 .

i

I

i
state
i
i Massachusetts
Connecticut
i
North Dakota
i
I New Hampshire
I

Iowa

percent

csat

yhat

79
81
6
75
5

896
897
1073
921
1093

847.3333
842.8571
1010.714
856.2856
1012.952

e I
I
48.666"3 I
54.14292 I
62.28567 I
64.71434 I
80.04758 I
+

Linear Regression Analysis

167

predict also derives other statistics from the most recently-fitted model.
Below are
some Predict options that can be
used after: anova or regress .
------------predict new

Predicted values ofy. predict new, xb means the
same thing (referring to Xb, the vector of predicted y
values).

. predict new, cooksd
predict new,

• predict DFxl,

covratio

Cook’s D influence measures.
COVRATIO influence measures; effect of each
observation on the variance-covariance matrix of
estimates.

dfbeta (xi) DFBETAs measuring each observation’s influence on the
coefficient of predictors/.

. predict new. dfits

DF/TS influence measures.

. predict new, hat

. predict new, resid

Diagonal elements of hat matrix (leverage).
Residuals.

. predict new, rstandard

Standardized residuals.

. predict new, rstudent

Studentized (jackknifed) residuals.

. predict new, stdf

Standard errors of predicted individual y, sometimes
called the standard errors of forecast or the standard
errors of prediction.

. predict new, stdp

Standard errors of predicted mean^y.

. predict new, stdr

Standard errors of residuals.

predict new, welsch

I

Welsch’s distance influence measures.
Further options obtain
predicted probabilities and expected
—amp!cuicicaproDaomnesand
expected values;
values; type
type help regress for
a list. All predict options create case statistics, which are new variables (like predicted
values and residuals) that have a value for each observation in the sample.
When using predict, substitute a new variable name of your choosing for new in the
commands shown above. For example, to obtain Cook’s D influence measures, type
. predict D, cooksd

Or you can find hat matrix diagonals by typing
. predict h, hat

The names of variables created by predict (such asy/mt, e, D, h) are arbitrary and are
invented by the user. As with other elements of Stata commands, we could abbreviate the
options to the minimum number of letters it takes to identify them uniquely. For example,
. predict e, resid

could be shortened to
. pre e,

re

168

Statistics with Stata

Basic Graphs for Regression
This section introduces some <elementary
*
graphs you can use to represent a regression model
or examine its fit. Chapter 7 describes
_j more specialized graphs that aid post-regression
diagnostic work.

In simple regression, predicted values lie on the line defined by the regression equation.
By plotting and connecting predicted values, we can make that line visible. The If it (linear
lit) command automatically draws a simple regression line.

. graph twoway Ifit csat percent
Ordinarily, it is more interesting to overlay a scatterplot on the regression line, as done in
Figured. I.
. graph twoway Ifit csat percent
I I
I I

scatter csat percent

, ytitle("Mean composite SAT score") legend(off)

o

Figure 6.1

2 -I|
£
i

!
i

0)


wo _
HO '

C^
co
0)

w
o
CL
E

8g-

£ 05
co
<D

o
CO

—J-

0

20

40
60
% HS graduates taking SAT

80

We could draw the same F'
Figure 6.1 graph “by hand” using the predicted values (yhat)
generated after the regression, and a command of the form
graph twoway mspline yhat percent, bands(50)
I|
scatter csat percent
II
, legend(off) ytitle("Mean composite SAT score")

The second approach is more work, but offers greater flexibility for advanced applications such
as conditional effect plots or nonlinear regression. Working directly with the predicted values
also keeps the analyst closer to the data, and to what a regression model is doing, graph
twoway mspline (cubic spline curve fit to 50 cross-medians) simply draws a straight line
when applied to linear predicted values, but will equally well draw a smooth curve in the case
of nonlinear predicted values.

Linear Regression Analysis

169

I
1

Residual-versus-predicted-values plots provide useful diagnostic tools (Figure 6.2). After
any regression analysis (also after some other models, such as ANOVA) we can automatically
draw a residual-versus-fitted (predicted values) plot just by typing
. rvfplot, yline(0)
o
o

Figure 6.2

o _
m

ro
ZJ
•g
V)
0)

or o

o

%

o
tn -

850

900

950
Fitted values

1000

1050

The “by-hand” alternative for drawing Figure 6.2 would be
. graph twoway scatter e yhat, yline(0)

Figure 6.2 reveals that our present model overlooks an obvious pattern in the data. The
residuals or prediction errors appear to be mostly positive at First (due to too-high predictions),
then mostly negative, followed by mostly positive residuals again. Later sections will seek a
model that better fits these data.
predict can generate two kinds of standard errors tor the predicted v values, which have
two different applications. These applications are sometimes distinguished by the names
confidence intervals’ and “prediction intervals”: A “confidence interval” in this context
expresses our uncertainty in estimating the conditional mean ot p at a given x value (or a given
combination ofx values, in multiple regression). Standard errors for this purpose are obtained
through
. predict SE,

stdp

Select an appropriate t value. With 49 degrees of freedom, for 95% confidence we should use
t = 2.01, found by looking up the / distribution or simply by asking Stata:
. display invttail(49,.05/2)
2.0095752

Then the lower confidence limit is approximately
.

generate lowl = yhat - 2.01*SE

and the upper confidence limit is

i

170

Statistics with Stata

'Bit:
. generate highl = yhat + 2.01*SE

Confidence bands in simple regression have an hourglass shape, narrowest at the mean of
x. We
graph these
an overlaid
ac thf*
We could
could graph
these using
using an
overlaid twowav
twoway command
command such
such as
the fniinxxrinrY
following,

graph twoway mspline lowl percent, cipattern(dash) bands(50)
11
mspline highl percent, cipattern(dash) bands(50)
11
mspline yhat percent, cipattern(solid) bands(50)
scatter csat percent
11
11
, legend(off) ytitle("Mean composite SAT score")

Ml

Shaded-area range plots (see help twoway_rarea ) offer a different way to draw such
graphs, shading the range between lowl and high]. Alternatively, Ifitci can do this
automatically, and take care of the confidence-band calculations, as illustrated in Figure 6.3.
Note the stdp option, calling for a conditional-mean confidence band (actually, the default).
. graph twoway Ifitci csat percent, stdp
||
scatter csat percent, msymbol(O)
II
, ytitle("Mean composite SAT score")
legend(off)
title("Confidence bands for conditional means (stdp)")
Figure 6.3

Confidence bands for conditional means (stdp)

o
o

o
o
o .

I

co
0)

'w
o

Q.

E

Oo
Oo CO

ro
<D

i
o
o
co

C?

20

40
60
% HS graduates taking SAT

80

The second type of confidence interval for regression predictions is sometimes called a
“prediction interval.” This expresses our uncertainty in estimating the unknown value ofy for
an individual observation with known x value(s). Standard errors for this purpose are obtained
by typing
. predict SEyhat,

stdf

Figure 6.4 (next page) graphs this prediction band using Ifitci withthe stdf option.
Predicting they values of individual observations as done in Figure 6.4 inherently involves
greater uncertainty, and hence wider bands, than does predicting the conditional mean of y
(Figure 6.3). In both instances, the bands are narrowest at the mean of*.

Linear Regression Analysis

171

• graph twoway Ifitci csat percent, stdf
11
scatter csat percent, msymbol(0)
11
, ytitlef’Mean composite SAT score")
legend(off)
title("Confidence bands for individual-case predictions (stdf)")

o Confidence bands for individual-case predictions (stdf)

Figure 6.4

O -

<D

8o
wo 1-2
<

O)
0)

s

c
ro
a>
S

o
oco

o

20

40
60
% HS graduates taking SAT

80

As with other confidence intervals and hypothesis tests in OLS regression, the standard
errors and bands just described depend on the assumption of independent and identically
istributed errors. Figure 6.2 has cast doubt on this assumption, so the results in Figures 6 3
and 6.4 could be misleading.

Correlations
correlate obtains Pearson product-moment correlations between enables.
.

correlate csat expense percent income high college

(obs=51)

csat
expense
percent
income
high
college

I

|
|
|
I
|
|

csat

expense

percent

income

high

college

1.0000
-0.4663
-0.8758
-0.4713
0.0858
-0.3729

1.0000
0.6509
0.6784
0.3133
0.6400

1.0000
0.6733
0.1413
0.6091

1.0000
0.5099
0.7234

1.0000
0.5319

1.0000

correlate uses only a jsubset
*
of the data that has no missing values on any of the
variables listed (with these particular variables, that does not matter became
,
. .
—i
—**v*. luunvi MwauoC no observations
have missing values). In this respect, the correlate command resembles regress,and
given the same variable list, they will use the same subset of the data. Analysts not employing

m i172

Statistics with Stata

regression or other multi-variable techniques, however, might prefer to find correlations based
upon all of the observations available for each variable pair. The command pwcorr
(pairwise correlation) accomplishes this, and can also furnish /-test probabilities for the null
hypotheses that each individual correlation equals zero.
.

pwcorr csat expense percent income high college,
I

csat

csat I

1.0000

+
I
I
expense I
I
I
percent I
I
I
income I
I
I
high |
I
I
college I
I
I

r
t

■ 1

expense

percent

income

sig

high

-0.4663
0.0006

1.0000

-0.8758
0.0000

0.6509
0.0000

1.0000

-0.4713
0.0005

0.6784
0.0000

0.6733
0.0000

1.000C

0.0858
0.5495

0.3133
0.0252

0.1413
0.3226

0.5099
C.0001

1 .0000

-0.3729
0.0070

0.6400
0.0000

0.6091
0.0000

0.7234
0.3000

0.5319
0.0001

college

1.0000

It is worth recalling here that if we drew many random samples from a population in which
a variables really had 0 correlations, about 5% of the sample correlations would nonetheless
test statistically significant” at the .05 level. Analysts who review many individual hypothesis
tests, such as those in a pwcorr matrix, to identify the handful that are significant at the 0>
level, therefore run a much higher than .05 risk of making a Type I error. This problem is called
the multiple comparison fallacy.” pwcorr offers two methods, Bonferroni and Sidak, for
adjusting significance levels to take multiple comparisons into account. Of these, the Sidak
method is more precise.
.

pwcorr csat expense percent income high college,

csat

expense

percent

income

high

college

I
+

csat

I
I
I
I
I
I
I
I
I
|
I
I
|
I
I
|
I

1.0000

expense

percent

income

high

-0.4663
0.0084

1.0000

-0.8758
0.0000

0.6509
0.0000

-0.4713
0.0072

0.6784
0.0000

0.6733
0.0000

i.:ooc

0.0858
1.0000

0.3133
0.3180

0.1413
0.9971

0.5099
0.0020

1.0000

-0.3729
0.1004

0.6400
0.0000

0.6091
0.0000

0.7234
0.0000

0.5319
0.0009

i

sidak sig
college

nnnn

X • V V V V

1.0000

■ii mu ii

hi

mi

Linear Regression Analysis

173

Comparing the test probabilities in the table above with those of the previous pwcorr
provides some idea of how much adjustment occurs. In general, the more variables we
correlate, the more the adjusted probabilities will exceed their unadjusted counterparts. See the
Base Reference Mannar?, discussion of oneway for the formulas involved.
correlate itself offers several important options. Adding the covariance option
produces a matrix of variances and covariances instead of correlations:
correlate w x y z, covariance

Typing the following after a regression analysis displays the matrix of correlations between
estimated coefficients, sometimes used to diagnose multicollinearity (see Chapter 7):
correlate, _coef

.

■The following command will display the estimated coefficients’ variance-covariance matrix,
from which standard errors are derived:
. correlate, _coef covariance

Peaison correlation coefficients measure how well an OLS regression line fits the data.
They consequently share the assumptions and weaknesses of OLS, and like OLS, should
generally not be interpreted without first reviewing the corresponding scatterplots. A
scatterplot matrix provides a quick way to do this, using the same organization as the
correlation matrix. Figure 6.5 shows a scatterplot matrix corresponding to the pwcorr
matrix given earlier. Only the lower-triangular half of the matrix is drawn, and plus signs are
used as plotting symbols. We suppress^ and x-axis labeling here to keep the graph uncluttered.
. graph matrix csat expense percent income high college,

half msymbol(+) maxis(ylabel(none) xlabel(none))

I

Mean
composite
SAT
score

i

X

Figure 6.5

i

Per pupa

I
% HS
graduates
taking
SAT

Median
household
income,
$1,000

rn-

* \-

.

t '*

-

J ‘

L

•4>

%

adults
HS
diploma
% adults
college
degree

I

174

Statistics with Stata

i-l

To obtain a scatterplot matrix corresponding to a ccorrelate

■ • matrix, from
correlation
which all observations having missing values have been dropped, we would need to qualify the
command. If all of the variables had some
some missine
missing values
values, we
we cnnld
could tvnA
type aa command such as
• graph matrix csat expense percent income high college if
csat <
& expense < • & income < . & high < . & college <

To reduce the likelihood of confusion and mistakes, it might make sense to create a new dataset
keeping only those observations that have no missing values:


keep if csat < . & expense < .
& college < .
save nmvstate

& income <

& high < .

In this example, we immediately saved the reduced dataset with a new name, so as to avoid
inadvertently writing over and losing the information in the old, more complete dataset. An
alternative way to eliminate missing values uses drop instead of keep:
. drop if csat >= . |
I college >= .

expense >= .

|

income >=

I high >= .

save nmvstate

In addition to Pearson correlations, Stata can also calculate several rank-based correlations
1 hese can be employed to measure associations between ordinal variables, or as an outlierresistant alternative to Pearson correlation for measurement variables. To obtain the Spearman
rank correlation between csat and expense, equivalent to the Pearson correlation if these
variables were transformed into ranks, type
.

spearman csat expense

<■

|i!

Number of obs =
Spearman's rho =

51
-0.42S2

Test of Ho: csat ar.d expense =re independent
Prob > I t I =
i.ooi-

Kendall’s r, (tau-a)and tb. (tau-b) rank correlations can be found easily for these data, although
with larger datasets their calculation becomes slow:
ktau csat expense

Number of obs =
Kendall's tau-a =
Kendall' s tau-b =
Kendal1 ' s score =
SE of score =

51
-0.2925
-0.2932
3 "3

123.095

(corrected for ties)

iest of Ho: csat and expense are independent
Prob > lz| =
0.0025
(continuity corrected)

For comparison, here is the Pearson correlation with its (unadjusted) P-value:

1

Linear Regression Analysis

175

• pwcorr csat expense, sig
I

csat

csat |
I
I
expense I
I
I

1.0000
-0.4663
0.0006

expense

1.0000

In this example, both spearman (-.4282) and pwcorr (-.4663) yield hitter correlations
than ktau (-.2925 or-.2932). All three agree that null hypotheses of no association can be
rejected.

Hypothesis Tests
Two types of hypothesis tests appear in regress output tables. As with other common
hypothesis tests, they begin from the assumption that observations in the sample at hand were
drawn randomly and independently from an infinitely large population.
1. Overall F test: The F statistic at the upper right in the regression table evaluates the null
hypothesis that in the population, coefficients on all the model’s x variables equal zero.
Individual t tests: The third and fourth columns of the regression table contain t tests for
each individual regression coefficient. These evaluate the null hypotheses that in the
population, the coefficient on each particular x variable equals zero.

2.

The t test probabilities are two-sided. For one-sided tests, divide these P-values in half.
In addition to these standard F and t tests, Stata can perform F tests of user-specified
hypotheses. The test command refers back to the most recent model-fitting command such
as anova or regress . For example, individual t tests from the following regression
report that neither the percent of adults with at least high school diplomas (high) nor the percent
with college degrees (college) has a statistically significant individual effect on composite SAT
scores.
.

regress csat expense percent income high college

Conceptually, however, both predictors reflect the level of education attained by a state’s
..Ioh^ and for some
------ --purposes we jHjgbt want to test thenu[1 hypothesis that both have
population,
effect. To do this, we Ibegin by repeating the multiple regression quietly , because we do
not need to see its full output again. Then-------use thej test command:
.

I

quietly regress csat expense percent income high college
test high college
( 1)
( 2)

high = 0.0
college = 0.0
F(

2/
45) =
Prob > F =

3.32
0.0451

176

Statistics with Stata

Unlike the individual null hypotheses, the joint hypothesis that coefficients on high and
college both equal zero can reasonably be rejected (P = .0451). Such tests on subsets of
coefficients are useful when we have several conceptually related predictors or when individual
coefficient estimates appear unreliable due to multicollinearity (Chapter 7).
test could duplicate the overall Ftest:
.

1

I

test expense percent income high college

test could also duplicate the individual-coefficient tests:
test expense

.

i

test percent
test income

and so forth. Applications of test more useful in advanced work include

I

1. Test whether a coefficient equals a specified constant. For example, to test the null
hypothesis that the coefficient on income equals 1 (Ho :P3 = 1), instead of the usual null
hypothesis that it equals 0 (Ho :P 3 = 0), type
.

test income = 1

2. Test whether two coefficients are equal. For example, the following command evaluates
the null hypothesis H0 :p 4 = p 5 *
test high = college

3.

Finally, test understands some algebraic expressions. We could request something like
the following, which would test Ho :p 3 = (P 4 + p 5) / 100:
.

test income = (high + college)/100

Consult help test for more information and examples.

Dummy Variables
Categorical variables can become predictors in a regression when they are expressed as one or
more {0,1} dichotomies called dummy variables.” For example, we have reason to suspect
that regional differences exist in states mean SAT scores. The tabulatse command will
generate one dummy variable for each category of the tabulated variable if we add a gen
(generate) option. Below, we create four dummy variables from the four-category variable
legion. The dummies are named regl, reg2, reg3 and reg4. regl equals 1 for Western states
and 0 for others; regl equals 1 for Northeastern states and 0 for others; and so forth.
.

tabulate region, gen(reg)

Geographica |
1 region |

-------------------+
West
N. East
South
Midwest

Freq.

Percent

Cum.

13
9

26.00
44 .00
76.00
100.00

|
|
|
|

12

26.00
18.00
32.00
24.00

Total |

50

100.00

16

•1

Linear Regression Analysis

177

describe regl-reg4

variable name

storage
type

display
format

byte
byte
byte
byte

regl
reg2
reg3
reg4

value
label

variable label

%8.0g
•%8.0g
%8.0g
%8.0g

region
region:
region:
region=

:Wes t
:N. East
:South
:Midwes t

tabulate regl

region

Wes |
t I

Freq.

Percent•

Cum.

37
13

74.00
26.00

74.00
100.00

50

100.00

region==N.
N. |
East |

Freq.

Percent

Cum.

0 I
1 I

41
9

82.00
18.00

82.00
100.00

Total |

50

100.00

0 I
1 I
—+
Total |
tabulate reg2

Regressing csat on one dummy variable, reg2 (Northeast), is equivalent to performing a
two-sample t test of whether mean csat is the same across categories of reg2. That is, is the
mean csat the same in the Northeast as in other U.S. states?
.

regress csat reg2

Source

I

Model
Residual

|
|

SS

df

MS

35191.4017
177769.978

1
48

35191.4017
3703.54121

Total |

212961.38

49

4346.15061

csat |

Coef.

Std. Err.

t

P>l 11

reg2 |
cons |

-69.0542
958.6098

22.40167
9.504224

-3.08
100.86

0.003
0.000

------------- +

Number of obs =
F( 1,
48) =
Prob > F
R-squared
=
Adj R-squared =
Root MSE

[95% Conf.

-114.0958
939.5002

50
9.50
0.0034
0.1652
0.1479
60.857

Interval]

-24.01262
977.7193

The dummy variable coefficient’s t statistic (t = -3.08, P = .003) indicates a significant
difference. According to this regression, mean SAT scores are 69.0542 points lower (because
b = -69.0542) among Northeastern states. We get exactly the same result (t = 3.08, P = .003)
from a simple t test, which also shows the means as 889.5556 (Northeast) and 958.6098 (other
states), a difference of 69.0542.

I

I

178

Statistics with Stata

i,h- ■

ttest csat, by(reg2)
Two sample t test with equal variances
Group |

Obs

Mean

Std. Err.

Std. Dev.

[95% Conf. Interval]

0 I
1 I

41
9

958.6098
889.5556

10.36563
4.652094

66.37239
13.95628

937.66
878.8278

979.5595
900.2833

combined I

50

946.18

9.323251

65.92534

927.4442

964.9158

69.0542

22.40167

24.01262

114.0958

diff

|

Degrees of freedom: 48
Ho : mean (0) - mean(l) = diff = 0

Ha : diff < 0
t =
3.0825
P < t =
0.9983

Ha: diff != 0
t =
3.0825
P > 111 =
0.0034

Ha: diff > 0
t =
3.0825
P > t =
0.0017

This conclusion proves spurious, however, once we control for the percentage of students
taking the test. We do so with a multiple regression of csat on both reg2 and percent.
.

regress csat reg2 percent

Source |

SS

df

MS

Model |
Residual |

174664.983
38296.3969

2
47

87332.4916
814.816955

Total |

212961.38

49

4346.15061

csat |

Coef.

Std. Err.

t

P> 111

reg2 |
percent |
_cons |

57.52437
-2.793009
1033.749

14.28326
.2134796
7.270285

4.03
-13.08
142.19

0.000
0.000
0.000

----- +

---- +

Number of obs =
F( 2,
47) =
Prob > F
=
R-squared
=
Adj R-squared =
Root MSE

50
107.18
0.0000
0.8202
0.8125
28.545

[95% Conf. Interval]
28.79016
-3.222475
1019.123

86.25858
-2.363544
1048.374

The Northeastern region variable reg2 now has a statistically significantj?osztzvc coefficient
(Z? = 57.52437, P < .0005). The earlier negative relationship was misleading. Although mean
SAT scores among Northeastern states really are lower, they are lower because higher
percentages of students take this test in the Northeast. A smaller, more “elite” group of
students, often less than 20% of high school seniors, take the SAT in many of the non-Northeast
states. In all Northeastern states, however, large majorities (64% to 81%) do so. Once we
adjust for differences in the percentages taking the test, SAT scores actually tend to be higher
in the Northeast.
To understand dummy variable regression results, it can help to write out the regression
equation, substituting zeroes and ones. For Northeastern states, the equation is approximately
predicted csat = 1033.7 + 57.5reg2 - 2.Spercent
= 1033.7 + 57.5 x 1 2.8percent
= 1091.2 - Z.Zpercent
1

Illi

Linear Regression Analysis

179

For other states, the predicted csat is 57.5 points lower at any given level ofpercent:

predicted csat = 1033.7 + 57.5 * 0 - 2.8percent
= 1033.7 - 2.8percent



w
I

Dummy variables in models such as this are termed “intercept dummy variables,” because they
describe a shift in the y-intercept or constant.
From a categorical variable with k categories we can
< define k dummy variables, but one of
these will be redundant. Once we know a state’s values on the West, Northeast, and Midwest
dummy variables, for example, we can already guess its value on the South variable. For this
reason, no more than k - 1 of the dummy variables — three, in the case of region — can be
included in a regression. If we try to include all the possible dummies, Stata will automatically
drop one because multicollinearity otherwise makes the calculation impossible.
.

regress

csat regl reg2 reg3 reg4 percent

Source

I

ss

df

MS

Model |
Residual |

181378.099
31583.2811

4
45

45344.5247
701.850691

Total

212961.38

49

4346.15061

|

csat |

regl
reg2
reg3
reg4
percent
_cons

|
|
I
|
|
|

Number of obs =
F( 4Z
45) =
Prob > F
R-squared
Adj R-squared =
Root MSE

50
64.61
0.0000
0.8517
0.8385
26.4 92

Coef.

Std. Err.

t

P> I 11

-23.77315
25.79985
-33.29951
(dropped)
-2.546058
1047.638

11.12578
16.96365
10.85443

-2.14
1.52
-3.07

0.038
0.135
0.004

-46.18162
-8.366693
-55.16146

-1.364676
59.96639
-11.43757

.2140196
8.273625

-11.90
126.62

0.000
0.000

-2.977116
1030.974

-2.115001
1064.302

[95% Conf.

Interval]

The model s fit
including R -, F tests, predictions, and residuals — remains essentially
the same regardless of which dummy variable we (or Stata) choose to omit. Interpretation of
the coefficients, however, occurs with reference to that omitted category. In this example, the
Midwest dummy variable (reg4) was omitted. The regression coefficients on regl. reg2. and
reg3 tell us that, at any given level of percent, the predicted mean SAT scores are
approximately as follows:
23.8 points lower in the West (regl = 1) than in the Midwest;
25.8 points higher in the Northeast (reg2 = 1) than in the Midwest: and
33.3 points lower in the South (reg3 = 1) than in the Midwest.
The West and South both differ significantly from the Midwest in this respect, but the Northeast
does not.

An alternative command, <areg , fits the same model without going through dummy
variable creation. Instead, it “absorbs” the effect of a ^-category variable such as region. The
model s fit, F test on the absorbed variable, and other key aspects of the results are the same
as those we could obtain through explicit dummy variables. Note that areg does not provide
estimates
coefficients
x__ of the
------22----- j on individual dummy variables, however.

180

.

Statistics with Stata

areg csat percent, absorb(region)

Number of obs =
F( 1,
45) =
Prob > F
R-squared
=
Adj R-squared =
Root MSE

p


csat |

Coef.

Std. Err.

t

P> I t |

percent |
_cons |

-2.546058
1035.445

.2140196
8.38689

-11.90
123.46

0.000
0.000

9.465

0.000

------ +

r

region |

F(3, 45) =

50
141.52
0.0000
0.8517
0.8385
26.492

[95% Conf. Interval]

-2.977116
1018.553

-2.115001
1052.337

(4 categories)

Although its output is less informative than regression with explicit dummy variables,
areg does have two advantages. It speeds up exploratory work, providing quick feedback
about whether a dummy variable approach is worthwhile. Secondly, when the variable of
haS
j 7jues’ creating dummies for each of them could lead to too many variables
or too large a model for our particular Stata configuration, areg thus works around the usual
limitations on dataset and matrix size.

J
S' ’i

Explicit dummy variables have other advantages, however, including ways to model
interaction effects. Interaction terms called “slope dummy variables” can be formed by
multiplying a dummy times a measurement variable. For example, to model an interaction
between Northeast/other region ^percent, we create a slope dummy variable called reg2perc.
. generate reg2perc = reg2 * percent
(1 missing value generated)

The new variable, reg2perc, equals percent for Northeastern states and zero for all other states.
We can include this interaction term among the regression predictors:
.

regress csat reg2 percent reg2perc

Source |

r

i:

ss^

df

MS

Number of obs =
F( 3,
46) =
Prob > F
R-squared
=
Adj R-squared =
Root MSE
=

50
= 2.27
0.0000
0.8429
0.8327
26.968

Model |
Residual |

179506.19
33455.1897

3
46

59835.3968
727.286733

Total |

212961.38

49

4346.15061

csat |

Coef.

Std. Err.

t

P> 111

[95% Conf. Interval]

-241.3574
-2.858829
4.179666
1035.519

116.6278
.2032947
1.620009
6.902898

-2.07
-14.06
2.58
150.01

0.044
0.000
0.013
0.000

-476.117
-3.26804
. 9187559
1021.624

reg2
percent
reg2perc
_cons

|
|
|
|

-6.597821
-2.449618
7.440576
1049.414

i':;

p
a

The interaction
= 2.58,
imeracnon is
is statistically
statistically significant
significant (t
(t =
2.58, P
P = .013). Because this analysis
includes both intercept (reg2) and slope (reg2perc) dummy variables, it is worthwhile to write
out the equations. The regression equation for Northeastern states is approximately

rLinear Regression Analysis

181

predicted csat = 1035.5 - 24l.4reg2 - l.Vpercent + 4.2reg2perc
= 1035.5 - 241.4 x 1 - 2.9percent + 4.2 x 1 x percent
- 794.1 4- X.lpercent
For other states it is

predicted csat — 1035.5 — 241.4 x 0 — 2.9percent + 4.2 x 0 x percent
= 1035.5 -2.9percent

An interaction implies that the effect of one variable changes, depending on the values of
some other variable. From this regression, it appears that percent has a relatively weak and
positive effect among Northeastern states, whereas its effect is stronger and negative among the
To visualize the results from a jslope-and-intercept

dummy variable regression, we have
several graphing possibilities. Without even fitting the model/we could ask
If it to do the
work as follows, with the results seen in Figure 6.6.
label define reg2 0 "other regions" 1 "Northeast"
label values reg2 reg2

.

• graph twoway Ifit csat percent

II
I I

scatter csat percent

, by(reg2, legend(off) note( ))
ytitle("Mean composite SAT score")
other regions

o

Figure 6.6

Northeast

T—

S
o

sg
0)

s

Q.

E

8 g.

c 05
<u
<D

o
OO

l—

0

20

40

60

80

0

20

% HS graduates taking SAT

I

40

60

80

I
Is
L-

I

a il

182

Statistics with Stata

Alternatively, we could fit the regression model, calculate predicted values, and use those
to make the a more refined plot such as Figure 6.7. The bands (50) options with both
mspline commands specify median splines based on 50 vertical bands, which is more than
enough to cover the range of the data.
. quietly regress csat reg2 percent reg2perc

. predict yhatl
. graph twoway scatter csat percent if reg2
I I

0
mspline yhatl percent if reg2 == 0, cipattern(solid)

bands(50)

11
11

scatter csat percent if reg2 ■ = 1, msymbol(Sh)
mspline yhatl percent if reg2 == 1, cipattern(solid)
bands (50)

II
, ytitle("Composite mean SAT score")
legend (order (1 3) label(1 "other regions")
label(3 "Northeast states") position(12) ring(0))
o
o

I



other regions

Figure 6.7



Northeast states

2

i•
Ho "
CO
c
(Q
<D

E

0)

o° "
E01
o
O

|
o
o
co

0

■I

20

40
60
% HS graduates taking SAT

80

Figure 6.7 involves four overlays: two scatterplots (csat vs.percent for Northeast and other
states) and two median-spline plots (connecting predicted values, yhatl, graphed against
percent for Northeast and others). The Northeast states are plotted as hollow squares,
msymbol (Sh). ytitle and legend options simplify they-axis title and the legend; in
their default form, both would be crowded and unclear.
Figures 6.6 and 6.7 both show the striking difference, captured by our interaction effect,
between Northeastern and other states. This raises the question of what other regional
differences exist. Figure 6.8 explores this question by drawing a csat-percent scatterplot with
different symbols for each of the four regions. In this plot, the Midwestern states, with one
exception (Indiana), seem to have their own steeply negative regional pattern at the left side of
the graph. Southern states are the most heterogeneous group.

Linear Regression Analysis

183

• graph twoway scatter csat percent if regl ==1
11
scatter csat percent if reg2 ==1, msymbol(Sh)
11
scatter csat percent if reg3
1, msymbol(T)
11
scatter csat percent if reg4
1, msymbol(+)
11
, legend(position(1) ring(0) label(1 "West")
label (2 "Northeast") label(3
.3 "South") label(4 "Midwest"))
o
o

Figure 6.8

• West
Northeast
▲ South + Midwest
<D

8o
Wo
HO

< T"

0)

<D
w
Q.

E
e

S05 I

<D

A

%

o o
00 L-r-

0

20

40
60
% HS graduates taking SAT

80

Automatic Categorical-Variable Indicators and Interactions

I

The xi (expand interactions) command simplifies the jobs of expanding multiple-category
variables into sets of dummy and interaction variables, and including these as predictors in
regression or other models. For example, in da^tstudent2.dta (introduced in Chapter 5) there
is a four-category variable year, representing a student’s year in college (freshman, sophomore,
etc.). W e could automatically create a set of three dummy variables by typing
. xi, prefix(ind) i.year
The three new dummy variables will be named indyear_2, indyearj, and indyearj. The
prefix () option specified the prefix used in naming the new dummy variables. If we typed
simply

. xi i.year
giving no prefix () option, the names Jyear_2,_Iyear_3, and _Iyear_4 would be assigned
(and any previously calculated variables with those names would be overwritten by the new
variables). Typing

. drop

I*

employs the wildcard * notation to drop all variables that have names beginning with J.

184

Statistics with Stata

fM
1

By default, xi omits the lowest value ofthe categorical variable when creating dummies
but this can be controlled. Typing the command
char _dta[omit]

prevalent

will cause subsequent xi commands to automatically omit the most prevalent category (note
the use of square brackets). char _dta [ ] preferences are saved with the data; to restore
the default, type
.

char __dta[omit]

Typing


.

r-

i

char year[omit]

3

would omit year 3. To restore the default, type
.

char year[omit]

xi can also create interaction terms involving two categorical variables, or one categorical
and one measurement variable. For example, we could create a set of interaction terms for year
and gender by typing
.

xi i.year*i.gender

From the four categories ofrear and the two categories ofgender, this xi command creates
seven new variables — four dummy variables and three interactions. Because their names all
begin with J, we can use the wildcard notation _/* to describe these variables:
.

J
■■

S'
i!

Z*

variable name

storage
type

_Iyear_2
_Iyear_3
_Iyear_4
_lger.der_l
_IyeaXger._2_l
_IyeaXgen_3_l
_IyeaXgen_4_l

byte
byte
byte
byte
byte
byte
byte

value
label

variable label

*6

year== 2
year== 3
year== 4
gender
year==2
year
3
year
4

*
*=
* t

-ac . wg

1
& gender
£ gender
& gender;

1
1
1

To create interaction terms for categorical variable year and measurement variable drink
(33-point drinking behavior scale), type
.

I
j

describe

xi i.year*drink

Six new variables result, three dummy variables for year, and three interaction terms
representing each of the rear dummies times drink. For example, for a sophomore student
Jyear2 - 1 and _IyeaXdrink_2 = 1 xdrink = drink. For a junior student, _Iyear2 = 0 and
JyearXdrinkJ = 0><drink =0: also Jyeari = 1 and JyeaXdrinkJ = 1 ^drink = drink, and so
forth.
describe _Iyea*

variable name
_Iyear_2
_Iyear_3
_Iyea r_4

storage
type

display
format

byte
byte
byte t

%8 . Og
%8 . Dg
%8v0g—

value
label

variable label



year==2
year==3
year«4

Linear Regression Analysis

_IyeaXdrink_2
_IyeaXdrink_3
_IyeaXdrink_4

float
float
float

%9.0g
%9.0g
%9.0g

(year
(year
(year

185

2) *drink
3) *drink
4) *drink

The real convenience of xi comes from its ability to generate dummy variables and
interactions automatically within a regression or other model-fitting command. For example,
to regress variable gpa (student’s college grade point average) on drink and a set of dummy
variables for year, simply type
xi:

regress gpa drink i.year

This command automatically creates the necessary dummy variables, following the same rules
described above. Similarly, to regress gpa on drink, year, and the interaction ofdrink and year,
type
xi :

regress gpa drink i.year*drink

i.year
i.year*drink

_Iyear_l-4
_IyeaXdrink_#

(naturally coded; _Iyear_l omitted)
(coded as above)

Source

SS

df

MS

Model |
Residual I

5.08865901
40.6630801

7
210

.726951288
.193633715

Total |

45.7517391

217

.210837507

gpa |

drink
_Iyear_2
_Iyear_3
_Iyear_4
drink
_IyeaXdrin~2
_IyeaXdrin~3
_IyeaXdrin~4
_cons

I
|
I
|
|
I
|
|
|

Number of obs =
F( 1,
210) =
Prob > F
=
R-squared
=
Adj R-squared =
Root MSE

218
3.75
0.0007
0.1112
0.0816
.44004

Coef.

Std. Err.

t

P> 111

-.0285369
-.5839268
-.2859424
-.2203783
(dropped)
.0199977
.0108977
.0104239
3.432132

.0140402
.314782
.3044178
.2939595

-2.03
-1.86
-0.94
-0.75

0.043
0.065
0.349
0.454

-.0562146
-1.204464
-.8860487
-.799868

-.0008591
.0366107
.3141639
. 3591114

.0164436
.016348
.016369
.2523984

1.22
0.67
0.64
13.60

0.225
0.506
0.525
0.000

-.0124179
-.0213297
-.0218446
2.934572

. 0524133
.043125
.0426925
3.929691

[95% Conf. Interval]

The xi: command can be applied in the same way before many other model-fitting
procedures such as logistic (Chapter 10). In general, it allows us to include predictor
(right-hand-side) variables such as the following, without first creating the actual dummy
variable or interaction terms.
i . catvar

Creates j-l dummy variables representing the j categories of
catvar.

i.catvarl*i.catvar2

Creates j-l dummy variables representing the j categories of
catvar1; k-\ dummy variables from the A: categories of catvar2;
and (/—1 )(A7—1) interaction variables (dummy x dummy).

Creates j-1 dummy variables representing the J categories of
catvar, and J-l variables representing interactions with the
measurement variable (dummy x measvar).
After any xi command, the new variables remain in the dataset.
i.catvar* measvar

I

186

Statistics with Stata

Stepwise Regression

I1

With the regional dummy variable terms we added earlier to the state-level data in states dta
we have many possible predictors of csat. This results in an overly complicated model, with
several coefficients statistically indistinguishable from zero.
.

regress csat expense percent income college high regl reg2
reg2perc reg3

ss

df

MS

Model [
Residual |

195420.517
17540.863

9
40

21713.3908
438.521576

Total |

212961.38

49

4346.15061

csat |

Coef .

Std. Er r.

t

P> 111

[95% Conf

Interval]

expense I
percent |
income i
college j
high |
regl I
reg2 |
reg2perc j
reg3 |
_cons |

-.0022508
-2.93786
-.0004919
3.900087
2.175542
-33.78456
-143.5149

. 0041333
.2302596
. 0010255
1.719409
1.171767
9.302983
101.1244
1.404483
12.54658
76.35942

-0.54
-12.76
-0.48
2.27
1.86
-3.63
-1.42
1.78
-0.70
10.99

0.589
0.000
0.634
0.029
0.071
0.001
0.164
0.082
0.487
0.000

-.0106045
-3.403232
-.0025645
.4250318
-.192688
-52.58659
-347.8949
-.3319506
-34.15679
684.8927

.006103
-2.472488
.0015806
7.375142
4.543771
-14.98253
60.86509
5.345183
16.55838
993.549

Source |

-------- +

y
»!

2.506616
-8.799205
839.2209

Number of obs =
F( 9,
40) =
Prob > F
R-squared
=
Adj R-squared =
Root MSE

50
49.51
0.0000
0.9176
0.8991
20.941

ii
We might: now try to simplify this model, dropping first that predictor with the highest t
probability (income, P= .634), then refittingthe model and deciding whetherto drop something
urther. Through this process of backward elimination, we seek a more parsimonious modelone that is simpler but fits almost equally well. Ideally, this strategy is pursued with attention
both to the statistical results and to the substantive or theoretical implications of keeping or
discarding certain variables.
For analysts in a hurry, stepwise methods provide ways to automate the process of model
selection. They work either by subtracting predictors from a complicated model, or by adding
predictors to a simpler one according to some pre-set statistical criteria. Stepwise methods
cannot consider the substantive or theoretical implications of their choices, nor can they do
much troubleshooting to evaluate possible weaknesses in the models produced at each step.
Despite their drawbacks, stepwise methods meet certain practical needs and have been widely
used.
J

I
Ill'

or automatic backward elimination, we issue a sw regress command that includes
a of our possible predictor variables, and a maximum P value required to retain them. Setting
the P-to-retain criteria as pr (. 05) ensures that only predictors having coefficients that are
significantly different from zero at the .05 level will be kept in the model.

i

£h

i

J

Linear Regression Analysis

187

sw regress csat expense percent income college high regl
reg2
■4

reg2perc reg3, pr(.O5)

p = 0.6341 >= 0.0500
p = 0.5273 >= 0.0500
P = 0.4215 >= 0.0500
p = 0.2107 >= 0.0500

begin with full model
removing income
removing reg3
removing expense
removing reg2

Source |

SS

df

MS

Model |
Residual |

194185.761
18775.6194

5
44

38837.1521
426.718624

Total

212961.38

49

4346.15061

|

Number of obs =
F( 5,
44) =
Prob > F
=
R-squared
Adj R-squared =
Root MSE

csat |

Coef.

Std. Err.

t

P> 111

I
I
I
I
I
I

-30.59218
-3.119155
.5833272
3.995495
2.231294
806.672

8.479395
. 1804553
.1545969
1.359331
.8178968
49.98744

-3.61
-17.28
3.77
2.94
2.73
16.14

0.001
0.000
0.000
0.005
0.009
0.000

regl
percent
reg2perc
college
high
cons

50
91.01
0.0000
0.9118
0.9018
20.657

[95% Conf. Interval]

-47.68128
-3.482839
.2717577
1.255944
. 5829313
705.9289

-13.50309
-2.755471
.8948967
6.735046
3.879657
907.4151

SW regress dropped first income, then reg3, expense, and finally reg2 before settling on
the final model. Although it has four fewer coefficients, this final model has almost the same
R2 (.9118 versus .9176) and a higher R’2a (.9018 versus .8991) compared with the earlier
version.

If, instead of a P-to-retain, pr (. 05), we specify a P-to-enter value such as pe (. 0 5),
then sw regress performs forward inclusion (starting with an “empty” or constant-only
model) instead of backward elimination. Other stepwise options include hierarchical selection
and locking certain predictors into the model. For example, the following command specifies
that the first term (xl) should be locked into the model and not subject to possible removal:
. sw regress y xl x2 x3, pr(.O5)

lockterml

The following command calls for forward inclusion of any predictors found significant at
the . 10 level, but with variables x4, x5, and x6 treated as one unit — either entered or left out
together:
sw regress y xl x2 x3 (x4 x5 x6) , pe(.10)

The following command invokes hierarchical backward elimination with a P = .20 criterion:
sw regress y xl x2 x3 (x4 x5 x6)

x7, pr(.20) hier

The hier option specifies that the terms are ordered: consider dropping the last term (x7)
first, and stop if it is not dropped. Ifx7 is dropped, next consider the second-to-last term (x4
x5 x6), and so forth.
Many other Stata commands besides regress also have stepwise variants that work in
a similar manner. Available stepwise procedures include the following:
sw clogit
Conditional (fixed-effects) logistic regression
sw cloglog

Maximum likelihood complementary log-log estimation

188

Statistics with Stata

sw cnreg

i

I

Censored normal regression
sw glm
Generalized linear models
sw logistic
Logistic regression (odds)
sw logit
Logistic regression (coefficients)
sw nbreg
Negative binomial regression
sw ologit
Ordered logistic regression
sw oprobit
Ordered probit regression
sw poisson
Poisson regression
sw probit
Probit regression
sw qreg
Quantile regression
s w regress
OLS regression
sw stcox
Cox proportional hazard model regression
sw streg
Parametric survival-time model regression
sw tobit
Tobit regression
Type help sw for details about the stepwise options and logic.

Polynomial Regression

I

Ii
i ■

Earlier in this chapter, Figures 6.1 and 6.2 revealed an apparently curvilinear relationship
between mean composite SAT scores (csai) and the percentage of high school seniors taking
the test (percent). Figure 6.6 illustrated one way to model the upturn in SAT scores at high
percent values: as a phenomenon peculiar to the Northeastern states. That interaction model
fit reasonably well (R 2a = .8327). But Figure 6.9 (next page), a residuals versus predicted
values plot for the interaction model, still exhibits signs of trouble. Residuals appear to trend
upwards at both high and low predicted values.
. quietly regress csat reg2 percent reg2perc
. rvfplot, yline(O)

Linear Regression Analysis
1

'

o
o

189

Figure 6.9

o
IO ■

<n
OJ



■u
‘w
0)

or

••

o

t •

o

ID "

850

900

950
Fitted values

1000

1050

Ch,apter 8 Presents a variety of techniques for curvilinear and nonlinear regression.
Curvilinear regression” here refers to intrinsically linear OLS regressions (for example,
regress ) that include nonlinear transformations of the original y or.v variables. Although
curvilinear regression fits a curved model with respect to the original data, this model remains
linear m the transformed variables. (Nonlinear regression, also discussed in Chapter 8, applies
non-OLS methods to fit models that cannot be linearized through transformation.)
One simple type of curvilinear regression, called polynomial regression, often succeeds in
fitting U or inverted-U shaped curves. It includes as predictors both an independent variable
and its square (andpossibly higher powers ifnecessary). Because the csat-percent relationship
appears somewhat U-shaped, we generate a new variable equal lopercent squared, then include
percent and percent2 as predictors of csat. Figure 6.10 graphs the resulting curve.
. generate percent2 = percent*!

.

regress csat percent percent2
Source |

ss

df

MS

Number of obs =
F(
2,
48) =
Prob > F
R-squared
Adj R-squared =
Root MSE

Model
Residual

|
|

193721.829
30292.6806

2
48

96860.9146
631.097513

Total

|

224014.51

50

4480.2902

csat

|

Coef.

Std. Er r.

t

P> 111

percent |
percent2 |
_cons |

-6.111993
.0495819
1065.921

. 6715406
.0084179
9.285379

-9.10
5.89
114.80

0.000
0.000
0.000

------------- +

-------- +

[95% Conf.
-7.462216
.0326566
1047.252

51
153.48
0.0 0 00
0.8648
0.8591
25.122

Interval]

-4.76177
. 0665072
1084.591

i

I

I

190

Statistics with Stata

I
• predict yhat2
(option xb assumed; fitted values)

ill

. graph twoway mspline yhat2 percent, bands(50)
11
scatter csat percent
11
, legend(off) ytitle("Mean composite SAT score")

i

o
o

Figure 6.10

i. I'
<D

§° I

Wq
HO

<Tcn

J

<p
"w

'll'

£o
cu

V-

I



Q.

E

“S-

<D

o
o 00 V

0

20

40
60
% HS graduates taking SAT

80

If we only wanted to see the graph, and did not need the regression analysis there is a

r

a curve similar to Figure 6.10 could have been obtained by typing
. graph twoway qfit csat percent
I I

j|M

r

O’
? Ji!

I

scatter csat percent

modlHnPFiS6
6J()TatChCS thC
Slightlybetterthanour interaction
Striking in a
h i °
-8591 versus -S327). Secanse the curvilinear pattern is now less
ndeXent dT r 11^
P'Ot
611^’ the usual assumption of
polyn^model
PlaUSlb’e
t0 thiS

Linear Regression Analysis

191

quietly regress csat percent percent2

. rvfplot, yline(O)

f
Figure 6.11
o

io

ro
"O
’</> o

a>

• •

CC

%
o
in -j___

850

900

950
Fitted values

1000

1050

In Figures 6.7 and 6.10, we have two alternative models for the observed upturn in SAT
scores at high levels of student participation. Statistical evidence seems to lean towards the
polynomial model at this point. For serious research, however, we ought to choose between
similar-fitting alternative models on substantive as well as statistical grounds. Which model
seems more useful, or makes more sense? Which, if either, regression model suggests or
corresponds to a good real-world explanation for the upturn in test scores at high levels of
student participation?

Although it can closely fit sample data, polynomial regression also has important statistical
weaknesses. The different powers of.v might be highly correlated with each other, giving rise
to multicollinearity. Furthermore, polynomial regression tends to track observations that have
unusually large positive or negative x values, so a few data points can exert disproportionate
influence on the results. For both reasons, polynomial regression results can sometimes be
sample-specific, fining one dataset well but generalizing poorly to other data. Chapter 7 takes
a second look at this example, using tools that check for potential problems.

Panel Data
Panel data, also called cross-sectional time series, consist of observations on i analytical units
or cases, repeated over t points in time. The Longitudinal/Panel Data Reference Manual
describes a wide range of methods for analyzing such data. Most of the relevant Stata
commands begin with the letters xt; type help xt for an overview. As mentioned in the
documentation, some xt procedures require time series or tsset data; see Chapter 13, or
type help tsset, for more about this step.

192

77-

i

ill

Statistics with Stata

This section considers the relatively simple case of linear regression with panel data,
accomplished by the command xtreg . Our example dataset, newfdiv.dta contains
information about the 10 census divisions of the Canadian p> v
i province of Newfoundland (Avalon
Peninsula, Burin Peninsula, and 8 others), for the years 1992-96.
Contains data from C:\data\netffdiv.dta
obs:
50

vars :
size:

7
2,250

variable name

storage
type

cendiv
divname
year
pop
unemp
outmig
tcrime

ii

Sorted by:

Newfoundland Census divisions
(source:
Statistics Canada)
18 Jul 2005 10:28

(99.9% of memory free)

display
format

byte
%9.0g
str20
%20s
int
%9.0g
double %9.0g
float
%9.0g
int
%9.0g
float
%9.0g

value
label

variable label

cd

Census Division
Census Division name
Year
Population, 1000s
Total unemployment. 1000s
Out-migration
Total crimes reported. 1000s

divname

year

pop

unemp

outmig

Avalon Peninsula
Avalon Peninsula
Avalon Peninsula
Avalon Peninsula
Avalon Peninsula

1992
1993
1994
1995
1996

259.587
261.083
259.296
257.546
255.723

58.56
52.23
44.81
39.35
38.68

6556
644 9
6907

Burin Peninsula
Burin Peninsula
Burin Peninsula
Burin Peninsula
Burin Peninsula

1992
1993
1994
1995
1996

29.865
29.611
29.327
26.898
28 . 126

9.5
9.18
8.41
7.12
6.61

874
928
584

cendiv

year

. list in 1/10

111
i-

<I?

1.
2.
3.
4.
5.

6.
7.
8.
9.
10 .

I cendiv
I-------I Avalon
I Avalon
I Avalon
I Avalon
I Avalon
I—
Burin
I
I
Burin
Burin
I
Burin
I
Burin
I

tcrime
26.211



21.039
20.201
19.536
21.268

I
I

1. 903
1.94
2.063
1 . 923

Figure 6.12 visualizes the panel data, graphing variations in the number ofcrimes reponed
each year for 9 of the 10 census divisions. Census division 1, the Avalon Peninsula, is by far
he largest in Newfoundland. Setting it temporarily aside by specifying if cendiv != 1
makes the remaining 9 plots in Figure 6.12 more readable. The imargin(l=3 r=3) option
in this example calls for left and right margins subplot margins equal to 3% of the graph width
giving more separation than the default.

Linear Regression Analysis

193

graph twoway connected tcrime year if cendiv != 1,
by(cendlvf note("")) xtitleC”') imargin (lef t=3 right=3)

Burin

S Coast

St Georg

Humber

Central

Bonavist

Notre D

NPen

Labrador

Figure 6.12

CO -

CM -

O
O
O

'

■o'

O)

€o
CL
£
co

co
04

<D

o
ro

H
co CM -

1992 1993 1994 1995 1996 1992 1993 1994 1995 1996 1992 1993 1994 1995 1996

The dataset contains 50 observations total. Because the 50 observations represent only 10
individual cases, however, the usual assumptions of OLS and other common statistical methods
do not apply. Instead, we need models with complex error specifications, allowing for both
unit-specific and individual-observation disturbances.
Consider the regression of v on two predictors, x and w. OLS regression estimates the
regression coefficients a, b, and c, and calculates the associated standard errors and tests,
assuming a model of the form
V,- — a -T bXj -r CM-, -r et
where the residuals for each observation, e i , are assumed to represent errors that have
independent and identical distributions. The i.i.d. errors assumption appears unlikely with
panel data, where the observations consist of the same units measured repeatedly.
A more plausible panel-data model includes two error terms. One is common to each of
the i units, but differs between units (u, ). The second is unique to each of the i, t obsen ations

yH = a + bxit + civf7 + W/ + e.
In order to fit such a model, Stata needs to know which variable identifies the i units, and
which variable is the time index t. This can be done within an xt command, or more
efficiently for the dataset as a whole. The commands iis (“z is”) and tis (‘7 is”) specify
the i and t variables, respectively. For newfdiv.dta, the units are census divisions (cendiv) and
the time index is^ear.

194

ifi

II
J i
'

!

Statistics with Stata

iis

cendiv

tis year

save, replace

Saving the dataset preserves the i and t specifications, so the iis and' tis
_L_ commands
are not required in a future session. Having set these variables, we can now fit a random-effects
(meaning that the common errors u, are assumed to be variable, rather than fixed) model
regressing tcrinie on unemp and pop.
xtreg

tcrime unemp pop,

re

Random-effects GLS regression
Group variable (i): cendiv
R-sq:

Number of obs
Number of groups

within
= 0.5265
between = 0.9717
overall = 0.9634

Obs per group: min =

5

avg =

5.0
5

max =

Random effects u i ~ Gaussian
corr(u_i, X)
= 0 (assumed)

Wald chi2(2)
Prob > chi2

H
tcrime |

’Ml

Std. Err.

z

P>|z|

[95* Conf. Interval]

unemp |
Pop I
_cons |

.1645266
.0558997
-.7264381

.0381813
.0073437
.301522

4.31
7.61
-2.41

0.000
0.000
0.016

.0896925
.0415062
-1.31741

sigma_u |
sigma_e I
rho |

.34458437
.42064667
.40157462

(traction of variance due to u_i)

■ i [J

Ub
n ■

H

r
|!

■■

705.54
0.0000

Coef.

------------------ 4.

Hit

50
10

.2393607
.0702931
.1354659

The xtreg output table contains regression coefficients, standard errors, t tests, and
confidence intervals that resemble those of an OLS regression. In this example we see that the
coefficient on unemp (. 1645) is positive and statistically significant. The predicted number of
crimes increases by .1645 for each additional person unemployed, if population is held
constant. Holding unemployment constant, predicted crimes increase by 5.59 with each 100person increase in population. Echoing the individual-coefficient z tests, the Wald chi-square
test at upper right (Z: = 705.54.#= 2. P< .00005) allows us to reject the joint null hypothesis
that the coefficients on unemp and pop are both zero.
This output table gives further information related to the two error terms. At lower left in
the table we find
sigma_u

standard deviation of the common residuals u,

sigma_e

standard deviation of the unique residuals e,

rho

fraction of the unexplained variance due to differences among the units (i.e.,
differences among the 10 Newfoundland census divisions).
VarfwJ/CVar^,] + Var[eJ)

At upper left the table gives three “R2 ” <statistics.
'
The definitions for these differ from the
true/?2 ofOLS. In the case of xtreg , the “R2 ” are based on fits between several kinds of
observed and predicted;/ values.



-

Il

Regression Diagnostics

I
It
|j

Do the data give us any reason to distrust our regression results? Can we find better ways to
specify the model, or to estimate its parameters? Careful diagnostic work, checking for
potential problems and evaluating the plausibility of key assumptions, forms a crucial step in
modem data analysis. We fit an initial model, but then look closely at our results for signs of
trouble or ways in which the model needs improvement. Many of the general methods
introduced in earlier chapters, such as scatterplots, box plots, normality tests, or just sorting and
listing the data, prove useful for troubleshooting. Stata also provides a toolkit of specialized
diagnostic techniques designed for this purpose.

i!

Autocorrelation, a complication that often affects regression with time series data, is not
covered in this chapter. Chapter 13, Time Series Analysis, introduces Stata’s library of time
series procedures including Durbin-Watson tests, autocorrelation graphs, lag operators and
time-series regression techniques.

4

Regression diagnostic procedures can be found under these menu selections:


Statistics - Linear regression and related - Regression diagnostics

i

Statistics - General post-estimation - Obtain predictions, residuals, etc., after estimation

h

Example Commands
1

The commands illustrated in this section all assume that you have just fit a model using either
anova or regress . The commands’ results refer back to that model. These followup
commands are of three basic types:
1. predict options that generate new variables containing case statistics such as predicted
values, residuals, standard errors, and influence statistics. Chapter 6 noted some key
options; type help regress for a complete listing.
2. Diagnostic tests for statistical problems such as autocorrelation, heteroskedasticity,
specification errors, or variance inflation (multicollinearity). Type help regdiag for
a list.

'
3. Diagnostic plots such as added-variable or leverage plots, residual-versus-fitted plots
residual-versus-predictorplots. and component-versus-residualplots. Again,typing help
regdiag obtains a full listing of regression and ANOVA diagnostic plots. General
graphs for diagnosing distribution shape and normality were covered in Chapter 2; type
help diagplots for a list of those.

iF
I

196

¥
Linear Regression Analysis

195

R2 within

Explained variation within units—defined as the squared correlation between
deviations of.y„ values from unit means (y-y,) and deviations of predicted
values from unit mean predicted values (y^-y,).

R2 between

Explained variation between units — defined as the squared correlation
between unit means (y,) and y t values predicted from unit means of the
independent variables.

R2 overall

Explained variation overall — defined as the squared correlation between
observed (yf7) and predicted (y,7) values.

Our example model does a very good job fitting the observed crimes overall (R2 = .96), and
also the variations among census division means (R2 = .97). Variations around the means
within census divisions are somewhat less predictable (R2 = .53).
The random-effects option employed for this example is one of several possible choices.

I

re

Generalized least squares (GLS) random-effects estimator; default

be

between regression estimator

fe

fixed-effects (within) regression estimator

mle

maximum-likelihood random-effects estimator

pa

population-averaged estimator

Consult help xtreg for further options and syntax. The Longitudinal/Panel Data
Reference Manual gives examples, references, and technical details.

I

>1
Regression Diagnostics

197

predict Options
. predict new, cooksd

Generates a new variable equal to Cook’s distance D, summarizing how much each
observation influences the fitted model.
. predict new, covratio

Generates a new variable equal to Belsley, Kuh, and Welsch’s COVRATIO statistic.
COVRATIO measures the ith case’s influence upon the variance-covariance matrix of the
estimated coefficients.
. predict DFxl, dfbeta(xl)

Generates DFBETA case statistics measuring how much each observation affects the
coefficient on predictor xl. The dfbeta command accomplishes the same thing more
conveniently, and in this example will automatically name the resulting statistics DFxl\
dfbeta xl

To create a complete set of DFBETAs for all predictors in the model, simply type the
command dfbeta without arguments.
. predict new, dfits

Generates DFITS case statistics, summarizing the influence of each observation on the
fitted model (similar in purpose to Cook’s D and Welsch’s IV).
Diagnostic Tests
I

I

dwstat

Calculates the Durbin-Watson test for first-order autocorrelation. Chapter 13 gives
examples of this and other time series procedures. See also:
help durbina
Durbin-Watson h statistic
help bgodfrey
Breusch-Godfrey LM (Lagrange multiplier) statistic
. hettest

Performs Cook and Weisberg’s test for heteroskedasticity. If we have reason to suspect
that heteroskedasticity is a function of a particular predictor xl, we could focus on that
predictor by typing hettest xl.
. ovtest,

rhs

Perfonns the Ramsey regression specification error test (RESET) for omitted variables. The
option rhs calls for using powers of the right-hand-side variables, instead of powers of
predicted y (default).
. vif

Calculates variance inflation factors to check for multicollinearity.
Diagnostic Plots

acprplot xl, mspline msopts(bands(7))

Constructs an augmented component-plus-residual plot (also known as an augmented
partial residual plot), often better than cprplot in screening for nonlinearities. The
options mspline msopts (bands (7)) call for connecting with line segments the
cross-medians of seven vertical bands. Alternatively, we might ask for a lowess-smoothed
curve with bandwidth 0.5 by specifying the options lowess Isopts (bwidth (. 5) ).

198

.

Statistics with Stata

avplot xl

Constructs an added-variable plot (also called a partial-regression or leverage plot) showing
the relationship between^ and xl, both adjusted for other x variables. Such plots help to
notice outliers and influence points.
avplots

Draws and combines in one image all the added-variable plots from the recent anova or
regress.
.

cprplot xl

Constructs a component-plus-residual plot (also known as a partial-residual plot) showing
the adjusted relationship betweeny and predictor xl. Such plots help detect nonlinearities
in the data.
Ivr2plot

Constructs a leverage-versus-squared-residual plot (also known as an L-Rplot).
rvfplot

Graphs the residuals versus the fitted (predicted) values of v.
.

rvpplot xl

Graphs the residuals against values of predictor xl.

SAT Score Regression, Revisited

-

i|


1

Diagnostic techniques have been described as tools for “regression criticism,” because they help
us examine our regression models for possible flaws and for ways that the models could be
improved. In this spirit, we return now to the state Scholastic Aptitude Test regressions of
Chapter 6. A three-predictor model explains about 92% of the variance in mean state SAT
scores. The predictors are percent (percent of high school graduates taking the test), percent?
(percent squared), and high (percent of adults with a high school diploma).
. generate percents = percent*!

. regress csat percent percents high
Source

|

Model |
Residual |

li-i

df

MS

Number of obs =
F(
3,
47) =
Prob > F
R-squared
Adj R-squared =
Root MSE

207225.103
16789.4069

3
47

69075.0343
357.221424

Total |

224014.51

50

4480.2902

csat |

Coef.

Std. Err.

t

P>l t|

percent |
percent2 |
high |
_cons |

-6.520312
.0536555
2.986509
844.8207

.5095805
.0063678
.4857502
36.63387

-12.80
8.43
6.15
23.06

0.000
0.000
0.000
0.000

------- +

I

ss

[95% Conf.

-7.545455
. 0408452
2.009305
771.1228

lil i

The regression equation is
predicted csat = 844.82 - 6.52percent + .05percent2 + 2.99high

51
193.37
0.0000
0.9251
0.9203
18.90

Interval]

-5.495168
.0664658
3.963712
918.5185

Regression Diagnostics

199

- four variables. As
-^interrelations among these
mOdel to fit *e visibly

11 ‘‘"SXws »«< ™"ssl°“

The scatterpH
noted in Chapter
curvilinear relate

and percentcs3t, half »s

yinbol ( + )

percents

Figure T ''

graph matrix

% HS I
graduates |

t3^ J

6000-

percenU

&

40002000-

90-

*
a**5
HS
dipt*03

A

80-

*♦

J

70 I

Mean
composite
SA‘
score

♦♦V*

60 -I
1100-f
1000-

r’

900-

800-{p
0

,

80

70

90

. The
:cond,
4 variance !)■
i of
,uld improve the

50

Several

It then perfe™ »
y equal zero. It
eRresston, we
model. With the C5^r
. ovtest
Ramsey RESET teS^
Ho: mode 1

not rei«'nU" W<“ ‘

. V- -J
fitted
— 3 of the
0,i.ng P°w“
!
variable
5
itted

n0 °

T.

values of csat

1.48

tPrror variance by
me assumption ofconstan A ( e cook and
’ > te
teStS
are linearly related 10 > (
st that in
t
7
standardized
resi^
thc
csat
regresston
su^
A heteroskedaStt^J d standardized resrdu ,
of constant vanance.
examining whether ^"ion and^Vh^oth^
example).
Weisberg 1994 for 4
the mil ..
csat
this instance we shot”
fitted values of

°-2319

gtO» >

. hettest

I

1

Cook-Weisberg te^
Ho: Constant
chi2(D
Prob >

s ..^roskedas
heteros

ticity

4.86
0.02'74

using

r

200

Statistics with Stata

“Significant” heteroskedasticity implies that our standard errors and hypothesis tests might be
invalid. Figure 7.2, in the next section, shows why this result occurs.
■f'

Diagnostic Plots
Chapter 6 demonstrated how predict can create new variables holding residual and
predicted values after a regress command. To obtain these values from our regression of
csut on percent, percent?, and high, we type the two commands:
. predict yhat3
. predict e3, resid

The new variables named e3 (residuals) andy/m/5 (predicted values) could be displayed in a
residual-versus-predicted graph by typing graph twoway scatter e3 yhat,
yline (0) . The rvfplot (residual-versus-fitted) command obtains such graphs in asingle
inreadinVsucl^pl?’2 'nCludeS 3 horizontal line at 0 (the residual mean), which helps
. rvfplot, yline(0)

Figure 7.2

o

CM

1 i

-Po -r

8

i
I '»

1

H

t.

H

I

i

9

%

o L

*

o

T ~l
850

900

950
Fitted values

1000

1050

Figure 7.2 shows residuals symmetrically distributed around 0 (symmetry is consistent with
the normal-en-ors assumption), and with no evidence of outliers or curvilinearity The
dispersion of the residuals appears somewhat greater for above-average predicted values ofy
however, which is why hottest earlier rejected the constant-variance hypothesis.
Kesidual-versus-fitted plots provide a one-graph overview of fhe regression residuals. For
more detailed study, we can plot residuals against each predictor variable separately through

(no?sh7wnrty^al’VerSUS'PrediCt°r” C°mrnandS- T° graph the residuaIs a8ainst Predictor high

>

Regression Diagnostics

.

201

rvpplot high

The one-variable graphs described in Chapter 3 can also be employed for residual analysis.
For example, we could use box plots to check the residuals for outliers or skew, or
quantile-normal plots to evaluate the assumption of normal errors.
Added-variable plots are valuable diagnostic tools, known by different names including
partial-regression leverage plots, adjusted partial residual plots, oradjusted variable plots. They
depict the relationship between y and one x variable, adjusting for the effects of other x
variables. If we regressed yonx2 andx3, and likewise regressed x7 onx2 andx3, then took the
residuals from each regression and graphed-these residuals in a scatterplot, we would obtain an
added-variable plot for the relationship betweeny andxf, adjusted forx2 and x3. An avplot
command performs the necessary calculations automatically. We can draw the adjustedvariable plot for predictor high, for example, just by typing
.

avplot high

Speeding the process further, we could type avplots to obtain a complete set of tiny
added-variable plots with each of the predictor variables in the preceding regression. Figure
7.3 shows the results from the regression of csat on percent, percent2, and high. The lines
drawn in added-variable plots have slopes equal to the corresponding partial regression
coefficients. For example, the slope of the line at lower left in Figure 7.3 equals 2.99 which
is the coefficient on high.
.

avplots

Figure 7.3

o

«—

^8x

x3-

8° '
*3-

3-5
0
e( percent | X)

-10

5

^io

coef = -6.5203116, se = .50958046. t = -12.8

-500

0
500
e( percent | X)

1000

coef = .05365555, se = .00636777. t ■ 8.43

8x

'5'

?■
-10

-5

0
e( high | X)

5

10

coef = 2.9865088, se = .48575023. t “ 6.15

Added-vanable plots help to uncover observations exerting a disproportionate influence on
the regression model. In simple regression with one x variable, ordinary scatterplots suffice for
this purpose. In multiple regression, however, the signs of influence become more subtle. An
observation with an unusual combination of values on several x variables might have high
leverage, or potential to influence the regression, even though none of its individual x values

202
|i

Statistics with Stata

IS

1
if i-

is unusual by itself. High-leverage observations show up in added-variable plots as points
horizontally distant from the rest of the data. We see no such problems in Figure 7.3, however.
If outliers appear, we might identify which observations these are by including observation
labels for the markers in an added-variable plot. This is done using the mlabel ( ) option,
just as with scatterplots. Figure 7.4 illustrates using state names (values of the string variable
state)as labels. Although such labels tend to overprint each other where the data are dense,
individual outliers remain more readable.
.

avplot high, mlabel(state)

Figure 7.4

• Iowa

8
• North DakOtegon

• llliftditpryland

• Wisconsin
• Virginia •Delaware
• Nebraska
• Sputh^akutmt
• California

• Tennessee

X
CD

Alaska

• Rhode Island * M«ss<



• FfqgSW^

of
• Kentucky
_^_^*<ouis?a?i§xas
'Arkansas
• Alabama
• NoftiGeereifoa

8

• New Hampshire
• Montaf^^Wir^l^0^

• Utah
• Wyoming

• MicNeaeda

• District of Columbia

• Mswsaett Carolina
• West Virginia

-10

• Idaho
• Ohio
• Arizona
• Indiana
• Oklahoma

-5

0
e( high | X )
coef = 2.9865088. se = .48575023, t = 6.15

5

10

Component-plus-residual plots, produced by commands of the form cprplot xl, take
a different approach to graphing multiple regression. The component-plus residual plot for
variable xl graphs each observation s residual plus its component predicted ffomx/,
et + bjxlj

against values of xl. Such plots might help diagnose nonlinearities and suggest alternative
functional forms. An augmented component-plus-residual plot (Mallows 1986) works
somewhat better, although both types often seem inconclusive. Figure 7.5 shows an augmented
component-plus-residual plot from the regression of csat on percent, percent?, and high.

I
Regression Diagnostics

203

acprplot high, lowess

f

o
co

Figure 7.5

ro o

1

V)

2
w

-to

c

o

h

o ‘O

eS

0)10

<
o
io

65

1

70

75
80
% over 25 w/HS diploma

85

The straight line in Figure 7.5 corresponds to the regression model. The curved line reflects
lowess smoothing based on the default bandwidth of .5, or half the data. The curve’s downturn
at far right can be disregarded as a lowess artifact, because only a few cases determine its
location toward the extremes (see Chapter 8). If more central parts of the lowess curve showed
a systematically curved pattern, departing from the linear regression model, we would have
reason to doubt the model’s adequacy. In Figure 7.5, however, the component-plus-residuals
medians closely follow the regression model. This plot reinforces the conclusion we reached
earlier from Figure 7.2, that the present regression model adequately accounts for all
nonlinearity visible in the raw data (Figure 7.1), leaving none apparent in its residuals.
As its name implies, a leverage-versus-squared-residuals plot graphs leverage (hat matrix
diagonals) against the residuals squared. Figure 7.6 shows such a plot for the csat regression.
To identify individual outliers, we label the markers with the values of state. The option
mlabsize (medsmall) calls for “medium small” marker labels, somewhat larger than the
default size of “small.” (See help testsizestyle fora list of other choices.) Most of
the state names form a jumble at lower left in Figure 7.6. but a few outliers stand out

■ Si

I

204

Statistics with Stata

Ivr2plot, mlabel(state) mlabsize(medsmall)

.

Figure 7.6

cn

• Connectici t

• Massachus etts

<u
om
ro
to

• Mississippi

•Utah
• Alaska

• New Hampshire
• Kentucky
’.A&oSrk

*

o

Carolina

'1I

S^e Island
ifla

Maryland

Jico

0

flbw--------------•District of Columbh
•Oregon
’ -• Nevada
• Michigan

• Illinois

.02

.04
.06
Normalized residual squared

.08

• Iowa
♦ Tennessee

.1

Lines in a leverage-versus-squared-residuals plot mark the means of leverage (horizontal
line) and squared residuals (vertical line). Leverage tells us how much potential for influencing
the regression an observation has, based on its particular combination of x values. Extreme v
values or unusual combinations give an observation high leverage. A large squared residual
indicates an observation with value much different from that predicted by the regression
model. Connecticut, Massachusetts, and Mississippi have the greatest potential leverage but
the model fits them relatively well. (This is not necessarily good. Sometimes, although not
here, high-leverage observations exert so much influence that they control the regression, and
it must fit them well.) Iowa and Tennessee are poorly fit, but have less potential influence.
Utah stands out as one observation that is both ill fit and potentially influential. We can read
its values by listing just this state. Because state is a string variable, we enclose the value
“Utah” in double quotes.
. list csat yhat3 percent high e3 if state

1.

I csat
I--------I 1031

yhat3

percent

high

1067.712

5

85.1

"Utah"

e3 I
----------------- I
-36.71239 |

Only 5% of Utah students took the SAT, and 85.1% of the state’s adults graduated from
high school. This unusual combination of near-extreme values on both x variables is the source
of the state’s leverage, and leads our model to predict mean SAT scores 36.7 points higher than
what Utah students actually achieved. To see exactly how much difference this one observation
makes, we could repeat the regression using Stata’s “not equal to” qualifier ! = to set Utah
aside.

Regression Diagnostics

..
.

regress

csat percent percent2 high if state

Source I

SS

MS

df

|
|

20109’.423
15214.1968

46

67032.4744
330.741235

Total

|

216311.52

49

4414.52082

csat

|

Std.

t

P> I 11

-13.44
9.02
6.74
22.87

0.000
0.000
0.000
0.000

percent |
percent2 I
high |
_cons |

-6.77=-06
.0563562
3.281’65
827.1159

.5044217
.0062509
.4865854
36.17138

"Utah”

Number of obs
F(
3,
46)
Prob > F
R-squared
Adj R-squared
Root MSE

Model
Residual

3

!=

205

=
=
=
=
=

50
202.67
0.0000
0.9297
0.9251
18.186

[95% Conf. Interval]

-7.794054
. 0437738
2.302319
754.3067

-5.763357
.0689387
4.26121
899.9252

In the n - 50 (instead of n = 51) regression, all three coefficients strengthened a bit because
we deleted an ill-fit obsenation. The general conclusions remain unchanged, however.
Chambers et al. (198?) and Cook and Weisberg (1994) provide more detailed examples and
explanations of diagnostic plots and other graphical methods for data analysis.

Diagnostic Case Statistics

I

I

Afterusing regress or anova, we can obtain a variety ofdiagnostic statistics through the
predict command (see Chapter 6 or type help regress ). The variables created by
predict are case statistics, meaning that they have values for each observation in the data.
Diagnostic work usually begins by calculating the predicted values and residuals.
There is some overlap in purpose among other predict statistics. Many attempt to
measure how much each obsenation influences regression results. “Influencing regression
results, however, could refer to several different things — effects on the r-intercept, on a
particular slope coefficient, on all the slope coefficients, or on the estimated standard errors,
for example. Consequently, we have a variety of alternative case statistics designed to measure
influence.
Standardized and studentized residuals (rstandard and rstudent) helpto identify
outliers among the residuals — observations that particularly contradict the regression model.
Studentized residuals have the most straightforward interpretation. They correspond to the t
statistic we would obtain by including in the regression a dummy predictor coded 1 for that
observation and 0 for all others. Thus, they test whether a particular observation significantly
shifts the ^-intercept.

i

Hat matrix diagonals ( hat ) measure leverage, meaning the potential to influence
regression coefficients. Observations possess high leverage when their x values (or their
combination ofx values) are unusual.

I

Several other statistics measure actual influence on coefficients. DFBETAs indicate by how
many standard errors the coefficient on xl would change if observation i were dropped from
the regression. These can be obtained fora single predictor, xl, in either of two ways: through
the predict option dfbeta(xl) or through the command dfbeta .

1'
206

Statistics with Stata

Cook’s D ( cooksd ), Welsch’s distance ( welsch ), and DFITS ( df its ), unlike
DFBETA, all summarize how much observation i influences the regression model as a whole
or equivalently, how much observation i influences the set of predicted values. CO VRATIO
measures the influence of the zth observation on the estimated standard errors. Below we
generate a full set of diagnostic statistics including DFBETAs for all three predictors. Note that
predict supplies variable labels automatically for the variables it creates, but dfbeta
does not. We begin by repeating our original regression to ensure that these post-regression
diagnostics refer to the proper (n = 51) model.
. quietly regress csat percent percent2 high
. predict standard,

rstandard

predict student, rstudent
. predict h, hat
. predict D, cooksd

kl

r !

I

.

predict DFITS,

.

predict

W,

df i ts

welsch

. predict COVRATIO, covratio

. dfbeta

I

DFpercent:
DFpercent2:
DFhigh:

I il

■h

.

describe standard - DFhigh

variable name
standard
student
h
D
DFITS
W
COVRATIO
DFpercent
DFpercent2
DFhigh

!1
!

Il <



7

storage
type
float
float
float
float
float
float
float
float
float
float

display
format

value
label

%9.0g
%9.0g
%9.0g
%9.0g
%9.0g
%9.0g
%9.0g
%9.0g
%9.0g
%9.0g

variable label

Standardized residuals
Studentized residuals
leverage
Cock’s D
Welsch distance
Covratio

summarize standard - DFhigh
Variable I
+
standard I

St-

DFbeta (percent)
DFbeta(percent2)
DFbeta(high)

student i
h I
D I
DFITS I

Obs

Mean

Std. Dev.

Min

Max

51
51
51
51
51

-.0031359
- . 00162
.0784314
. 0219941
- .0107348

1.010579
1.032723
.0373011
.0364003
.3064762

-2.099976
-2.182423
. 0336437
.0000135
-.896658

2.233379
2.336977
.2151227
.1860992
.7444486

51
51
51
51
51

- .089723
1.092452
.000938
-.0010659
-.0012204

2.278704
.1316834
.1498813
.1370372
.1747835

-6.854601
.7607449
- . 5067295
- .440771
-.6316988

5.52468
1.360136
.5269799
.4253958
.3414851

+
W
COVRATIO
DFpercent
DFpercent2
DFhigh

|
|
|
|
|

Ill

___________

Regression Diagnostics

207

summarize shows us the minimum and maximum values of each statistic, so we can
quickly check whether any are large enough to cause concern. For example, special tables
could be used to determine whether the observation with the largest absolute studentized
residual (student) constitutes a significant outlier. Alternatively, we could apply the Bonferroni
inequality and t distribution.table: max| student | is significant at level a if 111 is significant at
a/«. In this example, we have max| student | = 2.337 (Iowa) and n = 51. For Iowa to be a
s>gn>ficant outlier (cause a significant shift in intercept) at a = .05, t = 2.337 must be significant
. display
.00098039

.05/51

Stata’s ttail ( ) function can approximate the probability of 111 > 2.337, given dfh-K-1
= 51-3-1 =47:
. display 2*ttail(47,
. 02375138

2.337)

The obtained P-vaiue (P - .0238) is not below a/n = .00098, so Iowa is not a significant outlier
at a - .05.
n ^“‘‘^ ^iduals measure the /th observation’s influence on the?-intercept. Cook’s
• ’ k
'e SCh S distance 3,1 measure the /th observation’s influence on all coefficients
in the model (or, equivalently, on all n predicted;; values). To list the 5 most influential
observations as measured by Cook’s
type
. sort D
.

list state yhat3 D DFITS W in -5/1

state
47.
48 .
49.
50.

North Dakota
Wyoming
Tennessee
lows
Utah

yhat3
1036.696
1017.005
974.6981
1052.78
1067.712

D
. 0705921
.0789454
.111718
.1265392
. 1860992

□ FITS
. 5493086
-.5820746
. 6992343
. 7444486
- .896658

W
4.020527
-4.270465
5.162398
5.52468
-6.854601

The in -5/1 qualifier tells Stata to 1*list only the fifth-from-last (-5) through last
(lowercase letter “i1”) observations. Figure 7.7 shows one way to display influence graphically
symbols in a residual-versus-predicted plot are given sizes proportional to values of Cook’s D
through the “analytical weight" option [aweight = D]. Five influential observations stand
out, with large positive or negative residuals and high predicted csat values.

i

'J?

I

208

F

Statistics with Stata

• graph twoway scatter e3 yhat3 [aweight - D], msymbol(oh) yline(O)
o

hd.

Figure 7.7

o
CM

O

o

o
°o

o

o

_w
ro
Po

o

<0
0)

cr

o

o

o

%

o

8 O

o

I

o
o

T

850

900

950

1000

Fitted values

1050

Although they have different statistical rationales. Cook’s D, Welsch’s distance, andDFfTS
are closely related. In practice they tend to flag the same observations as influential. Figure
7.8 shows their similarity in the example at hand.
• graph matrix D W DFITS, half

Figure 7.8
Cooks
D

I
J

'

5

0

Welsch
distance

-5

1

h
0

Dfits

-1
0

.1

.2

-5

0

5

DFBETAs indicate how much each observation influences each regression coefficient.
Typing dfbeta after a regression automatically generates DFBETAs, for each predictor. In



Regression Diagnostics

209

this example, theyreceivedthenamesDFpercent(DFBETAforpredictorpercent'),DFpercent2,
and DFhigh. Figure 7.9 graphs their distributions as box plots.
.

graph box DFpercent DFpercent2 DFhigh,

legend(cols(3))

Figure 7.9

*9




—1

tn

r



■■■■ DFpercent

6®® DFpercent2

■■■ DFhigh

From left to right, Figure 7.9 shows the distributions oiDFBETAs for percent, percent?,
and high. (We could more easily distinguish them in color.) The extreme values in each plot
belong to Iowa and Utah, which also have the two highest Cook’s D values. For example,
Utah’s DFhigh = -.63. This tells us that Utah causes the coefficient on high to be .63 standard
errors lower than it would be if Utah were set aside. Similarly, DFpercent = .53 indicates that
with Utah present, the coefficient on percent is .53 standard errors higher (because the percent
regression coefficient is negative, “higher” means closer to 0) than it otherwise would be.
Thus, Utah weakens the apparent effects of both high and percent.
The most direct way to learn how particular observations affect a regression is to repeat the
regression with those observations set aside. For example, we could set aside all states that
move any coefficient by half a standard error (that is, have absolute DFBETAs of .5 or more):
.

regress csat percent percent2 high if abs(DFpercent)
abs(DFpercent2) < .5 & abs(DFhigh) < .5
Source I

-------------------- +

SS
ss

df

MS

<

.5 &

Number of obs =
F(
3,
44) =
Prob > F
R-squared
Adj R-squared =
Root MSE

Model
Residual

|
|

175366.782
11937.1351

3
44

58455.5939
271.298525

Total

|

187303.917

47

3985.18972

csat

|

Ccef.

Std. Err.

t

P> 111

percent
percent2
high
_cons

|
|
|
|

-6.510868
.0538131
3.35664
815.0279

.4700719
. 005779
.4577103
33.93199

-13.85
9.31
7.33
24.02

0.000
0.000
0.000
0.000

[95% Conf.

-7.458235
.0421664
2.434186
746.6424

48
215.47
0.0000
0.9363
0.9319
16.471

Interval]
-5.5635
.0654599
4.279095
883.4133

210

■3

I

fi

M-

Statistics with Stata

J“are^uI *nsPecti°n wiII reveal the details in which this regression table (based on n = 48)
differs from its n = 51 or n = 50 counterparts seen earlier. Our central conclusion — that mean
state SAT scores are well predicted by the percent of adults with high school diplomas and,
curvilinearly, by the percent of students taking the test — remains unchanged, however.
Although diagnostic statistics draw attention to influential observations, they do not answer
the question of whether we should set those observations aside. That requires a substantive
decision based on careful evaluation of the data and research context. In this example, we have
no substantive reason to discard any states, and even the most influential of them do not
fundamentally change our conclusions.
Using any fixed definition of what constitutes an “outlier,” we are liable to see more of
them in larger samples. For this reason, sample-size-adjusted cutoffs are sometimes
recommended for identifying unusual observations. After fitting a regression model with K
coefficients (including the constant) based on n observations, we might look more closely at
those observations for which any of the following are true:
leverage h > IK/n
Cook’s D > 4/n

['■

I’'

S'-

I

DFITS>2VK/n
Welsch’s IV> 3\/K
DFBETA >2/^
\COVRATIO- 1| > 2K/n
The reasoning behind these cutoffs, and the diagnostic statistics more generally, can be found
in Cook and Weisberg (1982,1994); Belsley, Kuh, and Welsch (1980); or Fox (1991).


ir

I

1

i

Multicollinearity
If perfect multicollinearity (linear relationship) exists among the predictors, regression
equations become unsolvable. Stata handles this by warning the user and then automatically
dropping one of the offending predictors. High but not perfect multicollinearity causes more
subtle problems. When we add a newx variable that is strongly related tox variables already
in the model, symptoms of possible trouble include the following:
1. Substantially higher standard errors, with correspondingly lower t statistics.
2. Unexpected changes in coefficient magnitudes or signs.
3. Nonsignificant coefficients despite a high 2?2.
Multiple regression attempts to estimate the independent effects of each x variable. There is
little information for doing so, however, if one or more of the x variables does not have much
independent variation. The symptoms listed above warn that coefficient estimateshave become
unreliable, and might shift drastically with small changes in the sample or model. Further
troubleshooting is needed to determine whether multicollinearity really is at fault and, if so,
what should be done about it.
Multicollinearity cannot necessarily be detected, or ruled out, by examining a matrix of
correlations between variables. A better assessment comes from regressing eachx on all of the
other x variables. Then we calculate 1 -R2 from this regression to see what fraction of the first

Regression Diagnostics

211

x variable’s variance is independent of the otherx variables. For example, about 97% of high's
variance is independent ofpercent and percent?'.

II

. quietly regress high percent percent2
. display 1 - e(r2)
.96942331

After regression, e(r2) holds the value of??2. Similar commands reveal that only 4% of
percent's variance is independent of the other two predictor variables:
. quietly regress percent high percent2
. display 1 - e(r2)
.04010307

This finding about percent and percent2 is not surprising. In polynomial regression or
regression with interaction terms, some x variables are calculated directly from other x
variables. Although strictly speaking their relationship is nonlinear, it often is close enough to
linear to raise problems of multicollinearity.

The post-regression command vif , for variance inflation factor, performs similar
calculations automatically. This gives a quick and straightforward check for multicollinearity.
. quietly regress csat percent percent2 high

. vif
Variable |

VIF

1/VIF

percent I
percent2 |
high |

24.94
24.78
1.03

0.040103
0.040354
0.969423

Mean VIF |

16.92

------- +

The 1/VIF column at right in a vif table gives values equal to 1 -R2 from the regression
of each x on the other x variables, as can be seen by comparing the values for high (.969423)
or percent (.040103) with our earlier display calculations. That is. 1.-VIF (or 1-/?;) tells
us what proportion of anx variable’s variance is independent of all the otherx variables. A low
proportion, such as the .04 (4% independent variation) of percent and percent?, indicates
potential trouble. Some analysts set a minimum level, called tolerance, for the 1/VIF value, and
automatically exclude predictors that fall below their tolerance criterion.
The VIF column at center in a vif table reflects the degree to which other coefficients’
variances (and standard errors) are increased due to the inclusion of that predictor. We see that
high has virtually no impact on other variances, but percent and percent? affect the variances
substantially. VIF values provide guidance but not direct measurements of the increase in
coefficient variances. The following commands show the impact directly by displaying
standard error estimates for the coefficient on percent, when percent? is and is not included in
the model.
. quietly regress csat percent percent2 high

I

L

. display __se [percent]
.50958046
. quietly regress csat percent high

. display _se[percent]
.16162193

kt,

■••I

212

Statistics with Stata

II
With pei cent2 included in the model, the standard error for percent is three times higher:
.50958046/. 16162193 = 3.1529166
This corresponds to a tenfold increase in the coefficient’s variance.
How much variance inflation is too much? Chatterjee, Hadi, and Price (2000) suggest the
following as guidelines for the presence of multi collinearity:
1. The largest VIF is greater than 10; or
2. the mean VIF is larger than 1.
With our largest VIFs close to 25, and the mean almost 17, the csat regression clearly meets
both criteria. How troublesome the problem is, and what, if anything, should be done about it,
are the next questions to consider.

Because/7erce«Z andpercent2 are closely related, we cannot estimate their separate effects
with nearly as much precision as we could the effect of either predictor alone. That is why the
standard error for the coefficient on percent increases threefold when we compare the
regression of csat on percent and high to a polynomial regression ofera/ on percent, percent2,
and high. Despite this loss of precision, however, we can still distinguish all the coefficients
from zero. Moreover, the polynomial regression obtains a better prediction model. For these
reasons, the multi col linearity in this regression does not necessarily pose a great problem, or
require a solution. We could simply live with it as one feature of an otherwise acceptable
model.

■SI i

i 1 ' :'
I '
!I

in

When solutions are needed, a simple trick called “centering” often succeeds in reducing
multicollinearity in polynomial or interaction-effect models. Centering involves subtracting the
mean from x variable values before generating polynomial or product terms. Subtracting the
mean creates a new variable centered on zero and much less correlated with its own squared
values. The resulting regression fits the same as an uncentered version. By reducing
multicollinearity, centering often (but not always) yields more precise coefficient estimates with
lower standard errors. The commands below generate a centered version of percent named
Cpercent, and then obtain squared values of Cpercent named Cpercent2.
summarize percent

Il i

Variable

|

Obs

Mean

Std. Dev.

Min

Max

percent

|

51

35.76471

26.19281

4

81

.

generate Cpercent = percent

r(mean)

.

generate Cpercent2 = Cpercent A2

S

fh

. correlate Cpercent Cpercent2 percent percent2 hiqh
(obs=51)

| •

I Cpercent Cperce~2

percent percent2

I
|
I
|
|
|

1.0000
0.9794
0.1413
-0.8758

------------------------------

ij

Cpercent
Cpercent2
percent
percent2
high
csat

J : *

>

1.0000
0.3791
1.0000
0.9794
0.1413
-0.8758

1.0000
0.3791
0.5582
-0.0417
-0.0428

1.0000
0.1176
-0.7946

csa t

high

csat

1.0000
0.0858

1.0000

Regression Diagnostics

213

JZ T'
Whereas percent and percent? have a near-perfect correlation with each other (r = .9794),
the centered versions Cpercent and Cpercent? are just moderately correlated (r = .3791).
Otherwise, correlations involving percent and Cpercent are identical because centering is a
linear transformation. Correlations involving Cpercent? are different from those with percent?,
however. Figure 7.10 shows scatterplots that help to visualize these correlations, and the
transformation’s effects.
.

graph matrix Cpercent Cpercent2 percent percent2 high csat,
half msymbol(+)

Figure 7.10
Cpercent
2000

Cpercent2

1000

108
% HS
graduates
taking

50

0
6000

4000
percent2

2000

98
80

% over
25 w/HS
diploma

70

1188
Mean
composite
SAT
score

1000
900

800

-50

500

0

1000

2000

50

100

70

2000 4000 600QE0

80

90

The/?2, overall Ftest, predictions, and many other aspects of a model should be unchanged
after centering. Differences will be most noticeable in the centered variable’s coefficient and
standard error.
.

regress csat Cpercent Cpercent2 high

Source |
Model
Residual

|
I

Total

I

ss

dr

MS

20"225.103
lc~89.407

3
47

69075.0343
357.221426

224014.51

50

4480.2902

Number of C-bS =
47) =
F( 3,
Prob > F
R-squared
Adj R-squared =
Root MSE

csat

I

Ooef.

Std. Err.

t

P> 111

Cpercent

I

-2.682362
.0536555
2.936509
680.2552

.1119085
. 0063678
.4857502
37.82329

-23.97
8.43
6.15
17.99

0.000
0.000
0.000
0.000

Cpercent2 I
high |
cons I

[95% Conf.

-2.907493
. 0408452
2.009305
604.1646

51
193.37
0.0000
0.9251
0.9203
18.90

Interval]

-2.457231
.0664659
3.963712
756.3458

In this example, the standard error of the coefficient on Cpercent is actually lower
(.111 9085 compared with .16162193) when Cpercent2 is included in the model. The t statistic
is correspondingly larger. Thus, it appears thatcentering did improve that coefficient estimate’s

214

I

Statistics with Stata

precision. The VIF table now gives less cause for concern: each of the three predictors has
more than 80% independent variation, compared with 4% for percent and percent2 in the
uncentered regression.
.

vif

I

VIE

1/VIF

Cpercent I
Cpercent2 |
high |

1.20
1.18

C .831528
. 8 4 6991
96942 3

Mean VIF I

1 .14

Variable

--------------- +

1.03

Another diagnostic table sometimes consulted to check for multicollinearity is the matrix
of correlations between estimated coefficients {not variables). This matrix can be displayed
after regress, anova , or other model-fitting procedures by typing
.

correlate,

coef

I Cpercent -perce-2
+----------------I
1.0000
|
-0.3893
1
200
|
-0.1700
0
240
|
0.2105
-0
151

Cpercent

Cpercent2
high
_cons

high

cons

1.0000
-0.9912

1.0000

High correlations between pairs of coefficients indicate possible collinearity problems.
By adding the option covariance , we can see the coefficients’ variance—covariance
matrix, from which standard errors are derived.
correlate,
I

Cpercent |
Cpercent2
high
_cons

I
|
|

__coef covariance
Cpercent Sperse'-2

high

cons

.012524
-.000277
.002241
-.009239
.00C322 .235953
.891126 -.051=17 -18.2105

1430.6

Position: 9 (160 views)