Appendix II: STATA Preliminary
STATA is a statistical software package that offers a large number of statistical and econometric estimation procedures. With STATA we can easily manage data and apply standard statistical and econometric methods such as regression analysis and limited dependent variable analysis to crosssectional or longitudinal data. STATA is widely used by analysts working with household survey data.

1. Getting Started 1.1. Starting STATA

Start a STATA session by double-clicking on the STATA icon in your desktop. The STATA computing environment comprises four main windows. The size and shape of these windows may be moved about on the screen. Their general look and description are shown below:

It is useful to have the Stata Results window be the largest so you can see a lot of information about your commands and output on the screen. In addition to these windows STATA environment has a menu and a toolbar at the top (to perform STATA operations) and a directory status bar at the bottom (that shows the current directory). You can use menu and toolbar to issue different
Poverty Manual, All, JH Revision of August 8, 2005 Page 170 of 218

STATA commands (like opening and saving data files), although most of the time it is more convenient to use the Stata Command window to perform those tasks. If you are creating a log file (see below for more details), the contents can also be displayed on the screen; this is sometimes useful if one needs to back up to see earlier results from the current session.
1.2. Opening a Dataset

You open a STATA dataset by entering following command in the Stata Command window: use hh

(or possibly use c:\intropov\data\hh, depending on where STATA has opened up; see below for more details). You can also click on File and the Open and then browse to find the file you need. STATA responds by displaying the following in the Stata Results window:
. use hh .

The first line repeats the command you enter and the second line with no error message implies that the command has been executed successfully. From now on, we will show only the Stata Results window to demonstrate STATA commands. The following points should be noted:
• •

STATA assumes the file to be in STATA format with an extension .dta. So, typing hh is the same as typing hh.dta. We can only have one data set open at a time in STATA. So if we open another data set ind.dta, we will be replacing hh.dta with ind.dta. Fortunately, STATA does not allow you do to this without warning, unless you type something like use ind.dta, clear. The above command assumes that the file hh.dta is in the current directory (shown by the directory status bar at the bottom). If that is not the case then you can do one of the following two things (assuming current directory is c:\stata\data and the file hh.dta is in c:\intropov): Type the full path of the data file:
. use c:\intropov\data\hh

•

Or make c:\intropov as the current directory and then open the file as before:
. cd c:\intropov\data . use hh .

If the memory allocated to STATA (which is 1,000K or 1M by default) is too little for the data file to be opened, as is typically the case when working with large household survey data sets, we will see an error message like the following:

Poverty Manual, All, JH Revision of August 8, 2005

Page 171 of 218

. use hh no room to add more observations r(901); .

The third line displays the code associated with the error message. All error messages in STATA have associated codes like this; further explanations are available in the STATA Reference Manuals. In this case we have to allocate more memory to STATA. The following commands allocate 30M to STATA and then try again to open the file:
. set memory 30m [This generates a table with information] . use hh .

Since the file opens successfully, allocated memory is sufficient. If you continue to get an error message, you can use a larger amount of memory, although this may slow down your computer somewhat. Note that the set memory command works only if no data set is open. Otherwise you will get following error message:

. use hh
. set memory 10m no; data in memory would be lost r(4);

You can clear the memory using one of the two commands: clear or drop _all. The following demonstration shows the first command:
. use hh . set memory 10m no; data in memory would be lost r(4); . clear . set memory 10m .

1.3 Saving a Data set

If you make changes in an open STATA data file and want to save those changes, you can do that by using the STATA save command. For example, the following command saves the hh.dta file:

Poverty Manual, All, JH Revision of August 8, 2005

Page 172 of 218

. save hh, replace file hh.dta saved

You can optionally omit the filename here (just save, replace is good enough). If you do not use the replace option STATA does not save the data but issues the following error message:
. save hh file hh.dta already exists r(602);

The replace option unambiguously tells STATA to overwrite the pre-existing original version with the new version. If you do NOT want to lose the original version, you have to specify a different filename in the save command, for instance using:
. save hh1 file hh1.dta saved

Notice that there is no replace option here. However, if a file named hh1.dta already exists then you have to either use the replace option or use a new filename.
1.4. Exiting STATA

An easy way to exit STATA is to issue the command: exit. However, if you have an unsaved data set open, STATA will issue the following error message:
. exit no; data in memory would be lost r(4);

To remedy this problem you can save the data file and then issue the exit command. If you really want to exit STATA without saving the data file, you can first clear the memory (using clear or drop _all command as shown before) and issue the exit command. You can also simplify the process by combining two commands:
. exit, clear

1.5. STATA Help

STATA comes with an excellent multivolume set of manuals. However, the on-computer help facility in STATA is extensive and very useful; and if you have access to the Web, an even larger set of macros and other useful information is available.

Poverty Manual, All, JH Revision of August 8, 2005

Page 173 of 218

•

From within STATA, if you know which command or keyword you want the help information about, you can the issue the command help followed by the command name or keyword. This command only works if you type the full command name or keyword unabbreviated. For example, the following will not work,
. help mem help for mem not found try help contents or search mem

but this will:
. help memory [output omitted]

•

If you can not recall the full command name or keyword, or you are not sure about which command you want you can use the command lookup or search followed by the command name or keyword. So following will work:
. search mem [output omitted]

•

This command will list all commands associated with this keyword and display a brief description of each of those commands. Then you can pick what you think is relevant and use “help” to obtain the specific reference. The STATA website (http://www.stata.com) has excellent help facilities, for example, Online Tutorial, Frequently Asked Questions (FAQ), etc.

1.6. Notes on STATA Commands

Here are some general comments about STATA commands:
• • •

STATA commands are typed in lower case. All names, including commands or variable names, can be abbreviated as long as there is no ambiguity. So describe, des or simply d do the same job as there is no confusion. In addition to typing, some keystrokes can be used to represent a few STATA commands or sequences. The most important of them are the Page-Up and Page-Down keys. To display the previous command in the Stata Command window, you can press the Page-Up key. You can keep doing that until the first command of the session appears. Similarly, the Page-Down key displays the command that follows the currently displayed command in the Stata Command window. Clicking once on a command in the Review window will put it into the Stata Command window; double clicking it will tell STATA to execute the command. This can be useful when commands need to be repeated, or edited slightly in the Stata Command window.

•

Poverty Manual, All, JH Revision of August 8, 2005

Page 174 of 218

2. Working with data files: looking at the content
From now on, we will mostly list the command(s) and not the results, to save space. To go through this exercise open the hh.dta file, as we will use examples extensively from this data file.

2.1. Listing the variables

To see all variables in the data set, use the describe command (fully or abbreviated):
. describe

This command provides information about the data set (name, size, number of observations) and lists all variables (name, storage format, display format, label). To see just one variable or list of variables use the describe command followed by the variable name(s):
. d hhcode vill storage display value variable name type format label variable label ---------------------------------------------------------------hhcode double %7.0f COMPLETE HOUSEHOLD CODE vill float %2.0f VILLAGE CODE

As you can see, the describe command shows also the variable type, its length and a short description of the variable (if available). The following points should be noted:
•

You can abbreviate a list of variables by typing only the first and last variable names, separated by a hyphen (-); the Variables window shows the order in which the variables are stored. For example, to see all variables from hhcode up to famsize, you could type:
. describe hhcode-famsize

•

The wildcard symbol (*) is helpful to save some typing. For example, to see all variables that start with "hh", you could type:
. describe hh*

You can abbreviate a variable or variable list this way in any STATA command (where it makes sense), not just in describe.

Poverty Manual, All, JH Revision of August 8, 2005

Page 175 of 218

2.2. Listing data

To see actual data stored in the variables we use the list command (abbreviated as l). If you type the command list all by itself STATA will display values for all variables and all observations, which is not desirable for any practical purpose (and you may need to use the Ctrl-Break combination to stop data from scrolling endlessly across the screen!). We usually want to see the data for certain variables and for certain observations. We achieve this by typing a list command with a variable list and with conditions, as shown in the following examples. The following command lists all variables of the first three observations:
. list in 1/3

Here STATA displays all observations starting with observation 1 and ending with observation 3. STATA can also display data as a spreadsheet. There are two icons in the toolbar called Data Editor and Data Browser (fourth and third from right). By clicking one, a new window will pop up and the data will be displayed as a table, with observations as rows and variables as columns. Data Browser will only display the data, whereas you can edit data with Data Editor. The commands edit and browse will also open the spreadsheet window. The following command lists household size and head's education for households headed by a female who is younger than 45:
. list famsize educhead if (sexhead==0 & agehead= (greater than or equal) >= (less than or equal) != or ~= (not equal)

Logical operators ~ (not) | (or) & (and)

You can use relational and logical operators in any STATA command (where it makes sense), not just in the list command.

Poverty Manual, All, JH Revision of August 8, 2005

Page 176 of 218

2.3. Summarizing data

The very useful command summarize (which may be abbreviated as sum) calculates and displays a few of summary statistics, including means and standard deviations. If no variable is specified, summary statistics are calculated for all variables in the data set. The following command summarizes the household size and education of the household head:
. sum famsize educhead

Any observation that has a missing value for the variable(s) being summarized is excluded from this calculation by STATA (missing values are discussed later). If we want to know the median and percentiles of a variable, we need to add the detail option (abbreviated d):
. sum famsize educhead, d

A great strength of STATA is that it allows for the use of weights. The weight option is useful if the sampling probability of one observation is different from another. In most household surveys, the sampling frame is stratified, where the first primary sampling units (often villages) are sampled, and conditional on the selection of primary sampling unit, secondary sampling units (often households) are drawn. Household surveys generally provide weights to correct for the sampling design differences and sometimes data collection problems. The implementation in STATA is straightforward:
. sum famsize educhead [aw=weight]

Here the variable weight has the information on the weight to be given to each observation and aw is a STATA option to incorporate the weight into the calculation. We will discuss the use of weights further in the chapter exercises given below. For variables that are strings, summarize will not be able to give any descriptive statistics except that the number of observations is zero. Also, for variables that are categorical (e.g. illiterate = 1, primary education = 2, higher education = 3), it can be difficult to interpret the output of the summarize command. In both cases, a full tabulation may be more meaningful, which we will discuss next. Many times we want to see summary statistics by group of certain variables, not just for the whole data set. Suppose we want to see mean family size, and education of household head, by region. We could use a condition in the sum command (for example, sum famsize educhead if region == 1 [aw=weight]), and so on for the other regions, but this is not convenient if the number of categories in the group is large. There is a simpler solution. First, sort the data by the group variable (in this case, region). You can check this by issuing describe command after opening each file. The describe command, after listing all

Poverty Manual, All, JH Revision of August 8, 2005

Page 177 of 218

the variables, indicates whether the data set is sorted by any variable(s). If there is no sorting information listed or the data set is sorted by a variable that is different from what you want it to be, you can use the sort command and then save the data set in this form. The following commands sort the data set by region and show summary statistics of family size and education of household head by region:
. sort region . by region: sum famsize educhead [aw=weight]

2.4. Frequency distributions (tabulations)

We often need frequency distributions and cross tabulations. We use the tabulate (abbreviated tab) command to do this. The following command gives the regional distribution of the households:
. tab region

The following command gives the gender distribution of household heads in region 1:
. tab sexhead if region==1

In passing, note the use of the = = sign here. It indicates that if the regional variable is identically equal to 1, then do the tabulation. We can use the tabulate command to show a two-way distribution. For example, we might want to check whether there is any gender bias in the education of household heads. We use the following command:
. tab educhead sexhead

To see percentages by row or columns we can add options to the tabulate command:
. tab region sexhead, col row

2.5. Distributions of descriptive statistics (table command)

Another very convenient command is table, which combines features of the sum and tab commands. In addition, it displays the results in a more presentable form. The following table command shows the mean of family size, and education of household head, by region:
. table region, c(mean famsize mean educhead)

Poverty Manual, All, JH Revision of August 8, 2005

Page 178 of 218

-----------------------------------------region | mean(famsize) mean(educhead) ----------+------------------------------Dhaka | 5.23 2 Chittagon | 5.82 3 Khulna | 5.03 3 Ragfhahi | 5.03 2 ------------------------------------------

The results are as we expected. But why is the mean of educhead displayed as an integer and not a fraction? This is because the educhead variable is stored as an integer number and STATA simply truncated numbers after the decimal. Look at the description of this variable.
. d educhead storage variable name type display format value label variable label

------------------------------------------------------------------------------educhead float %2.0f Education (years) of HH Head

We see that educhead is a float variable: its format (%2.0f) shows that its digits occupy 2 places and it has no digit after the decimal. You can force STATA to reformat the display. Suppose we want it display two places after the decimal, for a three-digit display. The following command shows that and subsequent table command:
. format educhead %3.2f . table region, c(mean famsize mean educhead) -----------------------------------------region | mean(famsize) mean(educhead) ----------+------------------------------Dhaka | 5.23 2.09 Chittagon | 5.82 3.14 Khulna | 5.03 2.91 Ragfhahi | 5.03 2.15 ------------------------------------------

This is much better. Formatting changes only the display of the variable, not the internal representation of the variable in the memory. The table command can display up to five statistics, and variables other than the mean (such as the sum or minimum or maximum). It is also possible to display two-way, threeway or even higher dimensional tables. Here is an example of a two-way table, which breaks down the education of the household head not just by region but also by sex of household head:

Poverty Manual, All, JH Revision of August 8, 2005

Page 179 of 218

. table region sexhead, c(mean famsize mean educhead) ---------------------| sex head | 1=male region | 0 1 ----------+----------Dhaka | 3.36 5.37 | 0.18 2.24 | Chittagon | 4.17 6.08 | 1.50 3.39 | Khulna | 4.18 5.11 | 1.36 3.05 | Ragfhahi | 3.70 5.13 | 0.00 2.31 ----------------------

2.5. Missing Values in STATA

In STATA, a missing value is represented by a dot (.). A missing value is considered larger than any number. The summarize command ignores the observations with missing values and the tabulate command does the same, unless forced to include missing values.
2.6. Counting observations

We use the count command to count the number of observations in the data set:
. count 519 .

The count command can be used with conditions. The following command gives the number of households whose head is older than 50:
. count if agehead>50 159 .

3. Working with data files: changing data set
3.1. Generating new variables

In STATA the command generate (abbreviated gen) creates new variables, while the command replace changes the values of an existing variable. The following commands create a new variable called oldhead, then set its value to 1 if the household head is older than 32 years and to 0 otherwise:
Poverty Manual, All, JH Revision of August 8, 2005 Page 180 of 218

. gen oldhead=1 if agehead>32 (98 missing values generated) . replace oldhead=0 if agehead sexhead = 1 Variable | Obs Mean Std. Dev. Min Max -------------+--------------------------------------------------educ | 2588 2.340417 3.324081 0 16 .log close

Poverty Manual, All, JH Revision of August 8, 2005

Page 186 of 218

What happens here is that STATA creates a text file named educm.log in the current directory and saves the summary output in that file. If you want the .log file to be saved in a directory other than current directory, you can specify full path of the directory in the .log creation command. You can also use the File button, followed by Log and Begin. If a .log file already exists, you could either replace it, with log using educm.log, replace, or append new output to it with log using educm.log, append. If you really want to keep the existing .log file unchanged, then you can rename either this file or the file in the .log creation command. In a .log file if you want to suppress portion of it, you can issue log off before that portion and log on command after that. You have to close a .log file before opening a new one, otherwise you will get an error message. If you find yourself using same set of commands repeatedly, you can save those commands in a file and run them together whenever you need it. These command files are called .do files, and are the STATA equivalent of “macros”. There are at least three good ways to create .do files: 1. Simply type the commands into a text file; label it educm.do (the .do suffix is important); and run the file using do educm in the Stata Command window. 2. Right click anywhere in the Review window; this will save all the commands that were used interactively. The file in which they were saved can be edited, labeled, and used as a .do file. 3. Use STATA’s built-in .do editor. It is invoked by clicking on the icon (the fifth from the right, at the top of the page). Commands may then be typed into the editor. These commands may be run by highlighting them and running them using the appropriate icon (the second from the right) within the .do editor. With practice this becomes a very quick and convenient way to work with STATA. Here is an example of a .do file: log using educm.log use ind sort hhcode save, replace use hh merge hhcode using ind tab _merge sort sexhead by sexhead:sum educ log close

The main advantages of using .do files instead of typing commands line by line are replicability and repeatability. With a .do file, one can replicate results that were worked on weeks or months before. And .do files are especially useful when sets of commands need to be repeated – for instance, with different data sets, or groups. There are certain commands that are useful in a .do file. We will discuss them from the following sample .do file:

Poverty Manual, All, JH Revision of August 8, 2005

Page 187 of 218

*This is a STATA comment that is not executed /*****This is a do file that shows some very useful commands used in do files. In addition, it creates a log file and uses some basic STATA commands ***/ #delimit ; set more 1; drop _all; cap log close; log using c:\intropov\logfiles\try1.log, replace; use c:\intropov\data\hh.dta ; describe ; list in 1/3 ; list hhcode famsize educhead if sexhead==2 & agehead